-- Logs begin at Tue 2022-05-24 14:30:08 UTC, end at Tue 2022-05-24 16:09:08 UTC. -- May 24 15:05:01.692010 dv-cinder-843151-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 24 15:05:01.700978 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 15:05:01 2022] *** May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: nodename: dv-cinder-843151-1-iscsi May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: machine: x86_64 May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: clock source: unix May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: pcre jit disabled May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: detected number of CPU cores: 4 May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: current working directory: / May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: detected binary path: /usr/bin/uwsgi-core May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: your processes number limit is 47680 May 24 15:05:01.702961 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: your memory page size is 4096 bytes May 24 15:05:01.703583 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: detected max file descriptor number: 2048 May 24 15:05:01.703583 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: lock engine: pthread robust mutexes May 24 15:05:01.703583 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: thunder lock: enabled May 24 15:05:01.703583 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 24 15:05:01.703583 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 15:05:01.747214 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: Python main interpreter initialized at 0x55b0e87219c0 May 24 15:05:01.747214 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: python threads support enabled May 24 15:05:01.747214 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: your server socket listen backlog is limited to 100 connections May 24 15:05:01.747214 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: your mercy for graceful operations on workers is 90 seconds May 24 15:05:01.747536 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: mapped 403077 bytes (393 KB) for 2 cores May 24 15:05:01.747610 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: *** Operational MODE: preforking *** May 24 15:05:01.747879 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: *** uWSGI is running in multiple interpreter mode *** May 24 15:05:01.747879 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: spawned uWSGI master process (pid: 80355) May 24 15:05:01.748062 dv-cinder-843151-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 24 15:05:01.749721 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: spawned uWSGI worker 1 (pid: 80365, cores: 1) May 24 15:05:01.749721 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: spawned uWSGI worker 2 (pid: 80366, cores: 1) May 24 15:05:01.749721 dv-cinder-843151-1-iscsi devstack@keystone.service[80355]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 15:05:07.361906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:05:07.361906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: from cryptography.utils import int_from_bytes May 24 15:05:07.362665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:05:07.362665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: from cryptography.utils import int_from_bytes May 24 15:05:07.363097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:05:07.363097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: from cryptography.utils import int_from_bytes May 24 15:05:07.363223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 15:05:07.363223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: from cryptography.utils import int_from_bytes May 24 15:05:07.410722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 15:05:07.410870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 15:05:07.410949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 15:05:07.411086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 15:05:07.411180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] command line args: [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 15:05:07.411262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] command line args: [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 15:05:07.411332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 15:05:07.411498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ================================================================================ {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 15:05:07.411577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 15:05:07.411678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ================================================================================ {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 15:05:07.411761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] admin_token = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.411928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] admin_token = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] config_dir = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] config_dir = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] config_source = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] debug = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] config_source = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] debug = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.412783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.413981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.414956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_config_append = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_config_append = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_dir = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_dir = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_file = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_file = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.415995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.416983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.417925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.418992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] publish_errors = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] publish_errors = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.419984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] standard_threads = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] standard_threads = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.420955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] use_journal = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] use_journal = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] use_json = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] use_json = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] use_stderr = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] use_stderr = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] use_syslog = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] use_syslog = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 15:05:07.421946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.422992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.423962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.424930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.425977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.426898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.427957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.428892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.429904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.430930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.431963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.432921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.433600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.434956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.435997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.436951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.437975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.438905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.439953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.440978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.441915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.442970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.443914 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.444996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.445928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.external = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.446969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.external = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.password = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.447901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] auth.token = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.password = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] auth.token = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.448965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.449081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.449195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.449791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.449908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.450947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.451967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.452995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.453927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.454974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.455979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.456934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.457983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.458905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.459853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.460895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.461932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.462997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.463904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.464811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.465514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.465650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.465767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.465881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.465995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.466912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.467969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.468942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.469986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.470903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.471934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.472963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.473999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.474917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.475956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.476941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.477970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.478939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.479912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] role.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.480976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] role.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] role.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] role.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.481907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.482943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.483975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.484794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.485937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.486973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.487893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.488991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.489916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.490960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.491999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.492951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.connection = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.connection = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.493909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.494947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.495991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.496915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80366) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 15:05:07.497723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 15:05:07.497723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80365) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 15:05:07.526340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.526499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.526578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.526710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.526790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.526864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.535886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.535977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.563747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.563971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.564054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.564271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.616601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.617202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.667650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.667878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.668530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.668765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.732970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.733886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.734011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.734102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.734224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.734433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.757262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.757492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.757715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.757834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.757917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.758920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.759128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.770480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.770945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.782925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.783900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.784027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.784108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.784234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.784334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80365) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.784470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80366) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 15:05:07.803257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.803461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:05:07.803795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.803877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.803995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:05:07.804078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:05:07.804258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.804368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.804538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.804621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 15:05:07.804792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.804876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.805043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.805121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.805295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.805543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.816570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.816845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.817847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.825928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.826507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.829197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.829476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.829755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.829835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.830033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.830926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.831975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.832063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.832241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.832530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.858262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.858934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.859959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.860235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.860664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.860943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.865449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.865723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.865997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.866126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.866402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.866717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.873735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.874012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.874403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.874682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.878431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.878824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.879385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.879917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.880960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.881224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.899335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.899607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.900219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.900515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.904236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.905174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.907838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.908128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.908792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.909084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.913177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.914220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.916768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.918020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.920562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.920859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.921148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.921434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.921705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.921786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.921976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.922087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.922377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.922657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.922937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.923213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.941265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.941542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.941803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.942056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.942535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.942813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.943077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.943332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.950542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.952038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.952239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.952540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.953770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.954044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.955556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.955833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.956104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.956412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.956683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.957081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.957356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.957630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.957911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.958182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.970138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.970406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.970680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.970962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.971234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.971449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.971551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.971726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.971809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.971998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.972280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.972588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.972864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.973140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.990272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.991193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.993508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.994432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:07.997312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.998253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.998899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:07.999852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.000536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.000824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.001097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.001481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.001772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.002052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.008435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.009706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.010021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.011287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.013888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.014180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.014459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.014729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.015146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.015434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.015714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.015981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.023714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.023998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.024400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.025020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.025298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.025677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.032201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80365) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.032506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.032784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.033051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.033314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.033525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80366) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 15:05:08.033607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.033799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.033890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.034969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80365) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.035924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.036188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.036534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.036806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.037070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80366) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 15:05:08.075048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b0e87219c0 pid: 80365 (default app) May 24 15:05:08.077502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55b0e87219c0 pid: 80366 (default app) May 24 15:05:08.078338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed0a8-f504-463c-a04c-f1c6717e529a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:08.078634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed0a8-f504-463c-a04c-f1c6717e529a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:08.078907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaed0a8-f504-463c-a04c-f1c6717e529a None None] PATH_INFO: `/v3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:08.083156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Tue May 24 15:05:08 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:15.838224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:15.838775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:15.838775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:15.843953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:15.849107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:15.849520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:15.849658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:15.917092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: INFO dbcounter [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] Registered counter for database keystone May 24 15:05:15.938296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG oslo_db.sqlalchemy.engines [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80365) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 15:05:15.944422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writer thread running {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 15:05:16.118464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG passlib.handlers.bcrypt [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80365) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 15:05:16.118818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG passlib.handlers.bcrypt [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80365) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 15:05:16.173809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:16.175370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:16.295752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 24 15:05:16.459616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:16.527678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: INFO dbcounter [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] Registered counter for database keystone May 24 15:05:16.547065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG oslo_db.sqlalchemy.engines [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80366) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 15:05:16.553198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writer thread running {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 15:05:16.673598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad35ab-fdb0-4b62-8956-e1376de1d319 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:16.677765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:16.678425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:16.678708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:16.678981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:16.687450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:16.739763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 15:05:16.740100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf653e8-341c-44ac-a64a-3b4b87f13938 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:16.749419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Tue May 24 15:05:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 15:05:16.755416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:16.769646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333693d-9467-43c5-9f2d-55b8345dd4a1 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:16.774014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:16.775121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:16.775409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:16.775684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] PATH_INFO: `/v3/projects/afcdba301526446ba48fe1e8c0bb21bc` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:16.777702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] RBAC: Authorizing `identity:get_project(project_id=afcdba301526446ba48fe1e8c0bb21bc)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:16.935503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING keystone.common.rbac_enforcer.enforcer [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 15:05:16.936149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941ec8eb-e58e-4678-a16c-48b885e6a769 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:16.940538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Tue May 24 15:05:16 2022] GET /identity/v3/projects/afcdba301526446ba48fe1e8c0bb21bc => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:19.156278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa67da-7db0-4330-90e0-ede1e3d35d33 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:19.156791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa67da-7db0-4330-90e0-ede1e3d35d33 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:19.156881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa67da-7db0-4330-90e0-ede1e3d35d33 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:19.160926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:19.166214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:19.166590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:19.166768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:19.186260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:19.187746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:19.195380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 15:05:19.217635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa67da-7db0-4330-90e0-ede1e3d35d33 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:19.229366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa67da-7db0-4330-90e0-ede1e3d35d33 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:19.233773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:19.234392 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:19.234672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:19.234945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:19.242458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:19.244218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116453c4-1bfe-4b98-ade3-5f008bb02a73 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:19.253377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Tue May 24 15:05:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 15:05:19.258741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:19.268147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849b8d8-a315-42fd-b8dd-1a303cf5871d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:19.272513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:19.273365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:19.273645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:19.273916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:19.275278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:19.277138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2785e-438f-4f3f-a034-f1abd4014da0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:19.278672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Tue May 24 15:05:19 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:21.525993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0585-00bc-4847-83ba-d93e12a4b767 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:21.526500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0585-00bc-4847-83ba-d93e12a4b767 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:21.526602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f0585-00bc-4847-83ba-d93e12a4b767 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:21.527873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:21.533183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:21.533594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:21.533739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:21.550064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:21.551217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:21.592942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42350-ed02-4aa5-a271-039a7114164c None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:21.593228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42350-ed02-4aa5-a271-039a7114164c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:21.593505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc42350-ed02-4aa5-a271-039a7114164c None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:21.595060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:21.599945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:21.600240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:21.600554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:21.631967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:21.640506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG passlib.handlers.bcrypt [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80366) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 15:05:21.640933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG passlib.handlers.bcrypt [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80366) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 15:05:21.653548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:21.676397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:21.691553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:21.693233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:21.756718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b28b3-399f-4aa0-bc95-5c2d98f1c5fe None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:21.790741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:21.791319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:21.791603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:21.791883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:21.808824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:21.829242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:21.861304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:21.862855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:21.888789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:21.967681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 7/14] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:05:21.972897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.036216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a70f6-204f-4874-b0e9-2b6a0aa51b81 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.038545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:22.079206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.080353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.080650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.080959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.083375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2956ece-f842-456f-9d8a-8287f30c2ffa None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.094700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:22.106736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:22.106872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:22.107668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:22.108508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:22.113787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.123741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.124561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.124841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.125113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.135133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.136643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:05:22.157124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:22.157936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:22.158718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:05:22.158718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:05:22.158718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:05:22.158718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:22.158718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 8/16] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:22.164029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.214018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b25b9f-882b-4b47-9653-a95090c6c002 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.255297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.255965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.256250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.256560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.257312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:22.257619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.259052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.265434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e3f335-ff05-47c9-85a8-80595bb188e6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.271284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:22.276825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.299582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.300241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.300553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.300826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.301543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:22.301849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.303256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.304259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:22.319996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:22.381030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7caf2-dfb3-4d06-ab88-d3e213b6c4e0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.417613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.418511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.418793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.419064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.420803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.422295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.423967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:22 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:22.429533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.466332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:22.536575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccffb54-94de-4570-9bba-95250608004c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.572039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.572858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.573136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.573410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.575189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.576702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab062ad3-ca90-49e0-92ed-2931e334ccd6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.578330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 10/20] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:22.584203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:22.686798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:22.694426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdbf13-f145-42ad-bdf2-2b5570a6fae0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:22.729920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:22.730628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:22.730908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:22.731181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:22.732022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.domain.id=default, user.id=3a944101ee294a37aa42cee1133a9032) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:22.732361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:22.733834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59289e6c-69d1-4b53-a3cd-ec21f0da94b8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:22.742662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 11/21] 10.222.0.45 () {56 vars in 1322 bytes} [Tue May 24 15:05:22 2022] GET /identity/v3/role_assignments?user.id=3a944101ee294a37aa42cee1133a9032&scope.domain.id=default&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:24.907187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e029-50b0-4b26-83d6-9ec21df06b01 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:24.907692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e029-50b0-4b26-83d6-9ec21df06b01 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:24.907796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92e029-50b0-4b26-83d6-9ec21df06b01 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:24.909107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 11/22] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:24.914327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:24.914716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:24.914897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:24.932504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:24.934236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:25.005136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 12/23] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:25.028294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e029-50b0-4b26-83d6-9ec21df06b01 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:25.133117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92e029-50b0-4b26-83d6-9ec21df06b01 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:25.134695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:25.171758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:25.172494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.172777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.173052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.184210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:25.185760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:05:25.204782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:25.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:25.206909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:05:25.206909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:05:25.206909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:05:25.206909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:25.206909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 12/24] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:25.211970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:25.309290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:25.324594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d55f7-84c4-40fc-b973-1a4bd2be1fe3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:25.375605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:25.376318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.376606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.376879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.377635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:25.377943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:25.379390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d001e2-a958-46ca-8b5a-bc7f70195195 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:25.409449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 13/25] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:25.415416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:25.548236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19fd81-9eca-4361-97f8-714f9feb8d51 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:25.582720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:25.583380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.583660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.583935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.584686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:25.586174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ce6f3-05af-449a-bd1e-8f76cd59d7e7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:25.590866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181def-faa7-4aa0-92d2-def8b4820722 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.591164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181def-faa7-4aa0-92d2-def8b4820722 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.591449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2181def-faa7-4aa0-92d2-def8b4820722 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.592865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 14/26] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:25.597813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4d8f-e970-4430-a76f-615578976b98 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.598098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4d8f-e970-4430-a76f-615578976b98 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.598375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda4d8f-e970-4430-a76f-615578976b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.617912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5cda4d8f-e970-4430-a76f-615578976b98 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:25.619108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5cda4d8f-e970-4430-a76f-615578976b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:25.697898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 15/27] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:25.699143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0563-19c9-4164-88e6-7500fa272f6e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.699431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0563-19c9-4164-88e6-7500fa272f6e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.699710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae0563-19c9-4164-88e6-7500fa272f6e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.705181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 16/28] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:25.710576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.710868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.711152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.722212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 13/29] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:05:25 2022] POST /identity/v3/projects => generated 301 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:25.734608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c97dc-45cb-4e67-bfe8-9a25287ab542 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.734913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c97dc-45cb-4e67-bfe8-9a25287ab542 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.735173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c97dc-45cb-4e67-bfe8-9a25287ab542 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.736549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 14/30] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:25.738084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:25.739370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:25.742322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:25.742610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:25.742889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:25.779479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:25.801522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:25.803038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:25.859249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 17/31] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:25.860119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:25.879113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:25.940363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 15/32] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:25.941026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.023645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5048-c6bd-4e87-880a-9a76b3ab0958 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.087198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4c341-e326-4c6c-b6a1-c36f7f68fa46 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.092962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.093779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.094060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.094337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.109040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:26.118740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:26.119593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:26.120352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:26.123566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 18/33] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:26.124771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.141327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.142512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.142800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.143350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.144124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.146332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.151192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:26.169291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 16/34] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 15:05:25 2022] POST /identity/v3/roles => generated 219 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:26.171639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.215861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:26.261673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cd6ea-c4b9-4e1a-b88a-824d5b2b2933 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.297798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586757fd-e99c-4edc-bc1d-7f9620ff8efc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.303496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.304435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.304891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.305368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.306259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.308082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.340772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.341712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.342129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.342609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.343607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:26.344058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.345836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.362033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 17/35] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:26.370281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.454755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 19/36] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 15:05:26 2022] POST /identity/v3/roles => generated 225 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:26.490307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:26.548905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db84c32-328f-4912-8574-cc84117abaf7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.602604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.603905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.604411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.605698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.607312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.609089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c3fa4-346b-4816-a182-e2640ab2479b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.611252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 18/37] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:26 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:26.617563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.648096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e207f8-76ab-4201-8696-89440221ece4 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.648440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e207f8-76ab-4201-8696-89440221ece4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.648729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e207f8-76ab-4201-8696-89440221ece4 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.652580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 19/38] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:26.658559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c814-114e-45c4-b20d-142e4b715ea5 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.658851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c814-114e-45c4-b20d-142e4b715ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.659130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39c814-114e-45c4-b20d-142e4b715ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.678133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fe39c814-114e-45c4-b20d-142e4b715ea5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:26.679396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fe39c814-114e-45c4-b20d-142e4b715ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:26.772120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0bada-d464-4639-8d5c-f187dabb4149 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.778226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 20/39] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:26.780917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cdd57-aec0-46f0-8be6-2ca6625d0b42 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.780917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cdd57-aec0-46f0-8be6-2ca6625d0b42 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.780917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2cdd57-aec0-46f0-8be6-2ca6625d0b42 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.781299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 21/40] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:26.786649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.787114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.787344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.801346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:26.804853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:26.806191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:26.810594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.811688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.812263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.812575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.813868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.815447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.817103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 20/41] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:26.817864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.861786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:26.879293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 22/42] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:26.880076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.916159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458679ea-b489-427d-8151-f0d1b81c224b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:26.950832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:26.951618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:26.951903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:26.952181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:26.953386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:26.954871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:26.956509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 21/43] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:26.957254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:26.987892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296a3c6-a7d8-474b-977a-5b4a29dbd6c6 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:27.017467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:27.022397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:27.023268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:27.023555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:27.023838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] PATH_INFO: `/v3/domains/default/users/3a944101ee294a37aa42cee1133a9032/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:27.026167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3a944101ee294a37aa42cee1133a9032, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:27.027653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:27.043093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 23/44] 10.222.0.45 () {58 vars in 1361 bytes} [Tue May 24 15:05:26 2022] PUT /identity/v3/domains/default/users/3a944101ee294a37aa42cee1133a9032/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:27.043879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:27.057874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc61a-72c5-4b97-9c40-08229c0f7a3d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:27.097003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:27.097770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:27.098060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:27.098401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:27.099218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:27.100694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e1a6bd-fc9e-456b-87fd-ee19286da336 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:27.108217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:27.123592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 22/45] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 15:05:26 2022] POST /identity/v3/roles => generated 223 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:27.175383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c205c86c-5c19-4869-98ca-5a3f694befa1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:27.215059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:27.215817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:27.216105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:27.216411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:27.217156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:27.218691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc7181-4939-451f-b7fc-3d74df675bda None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:27.252101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 24/46] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:05:27 2022] POST /identity/v3/projects => generated 312 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:29.250836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0fe7-39d2-46af-bf31-5670e8d6f7bf None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.251335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0fe7-39d2-46af-bf31-5670e8d6f7bf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.251413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0fe7-39d2-46af-bf31-5670e8d6f7bf None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.252944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 23/47] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:29.258088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8574-cfd6-4cd2-94b6-590db8ebda97 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.258466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8574-cfd6-4cd2-94b6-590db8ebda97 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.258770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dd8574-cfd6-4cd2-94b6-590db8ebda97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.276377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a0dd8574-cfd6-4cd2-94b6-590db8ebda97 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:29.277619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a0dd8574-cfd6-4cd2-94b6-590db8ebda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:29.345460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 25/48] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:29.368614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc0fe7-39d2-46af-bf31-5670e8d6f7bf None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.463585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff59d29-6da3-47d8-89a4-fabade807a36 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.463879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff59d29-6da3-47d8-89a4-fabade807a36 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.464722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff59d29-6da3-47d8-89a4-fabade807a36 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.466181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 26/49] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:29.467828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc0fe7-39d2-46af-bf31-5670e8d6f7bf None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.471142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.471427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.471704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.488827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:29.490021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:29.494169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:29.506795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.507902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.508180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.512910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:29.518536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:29.526148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:29.527568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:29.528545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 24/50] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:29.533453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.558044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 27/51] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:29.580730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.630540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f54d23-25c6-4840-b3dd-326748f742fd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.663559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.664223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.664537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.664815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.665569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:29.665881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:29.667281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:29.675780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 25/52] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:29.679449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae3121-bf7b-4682-8a9f-b7d4a9a85edf None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.681201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.712456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.713252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.713536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.713822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.715581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:29.717093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:29.718700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 28/53] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:29.724077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.725950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:29.751962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:29.789513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464abb51-d6be-4487-a8f8-f0169e40f1e4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.822746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.824140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.824459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.824742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.825960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:29.827423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:29.827586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ec678-fffb-416d-bd9e-b1a26e166142 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.829091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 26/54] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:29.834424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:29.861678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.862394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.862686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.862965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.863673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:29.865162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:29.890098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 29/55] 10.222.0.45 () {60 vars in 1140 bytes} [Tue May 24 15:05:29 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:05:29.937420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b5664-1950-41a8-b4d5-c89fc60f4cb7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:29.953453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:29.975642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:29.976476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:29.976762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:29.977043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:29.978254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:29.980008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad02597-caf5-49cb-8b38-1ec5bfee7adf None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:29.981778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 27/56] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:29.987735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:30.069301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:30.095007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f351ae-15f4-4597-9c94-17531fa2f7c4 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:30.127993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:30.128683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:30.128966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:30.129240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:30.130071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.domain.id=default, user.id=3a944101ee294a37aa42cee1133a9032) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:30.130379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:30.131829 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da48a720-26a2-471a-ba7f-55b956200772 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:30.139232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 30/57] 10.222.0.45 () {56 vars in 1322 bytes} [Tue May 24 15:05:29 2022] GET /identity/v3/role_assignments?user.id=3a944101ee294a37aa42cee1133a9032&scope.domain.id=default&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:32.078104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3815b-e419-4f88-b2bc-9f00bbd096a4 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:32.078524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3815b-e419-4f88-b2bc-9f00bbd096a4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:32.078673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3815b-e419-4f88-b2bc-9f00bbd096a4 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:32.080052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 28/58] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:32.085141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:32.085528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:32.085710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:32.102081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:32.103259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:32.167832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 31/59] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:32.191903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3815b-e419-4f88-b2bc-9f00bbd096a4 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:32.290691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3815b-e419-4f88-b2bc-9f00bbd096a4 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:32.323090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:32.323839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:32.324124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:32.324433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:32.325597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:32.327063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35324120-5b0a-42ee-9425-e1cf3a611670 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:32.328665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 29/60] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:32.333855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:32.401052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:32.430040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae7cf6-a1ec-4d17-a7ea-bbbdc308c249 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:32.463521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:32.464187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:32.464504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:32.464787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:32.465495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:32.466966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046c253-76ac-4ef3-b47c-c43e6b62c55d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:32.492286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 32/61] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:32 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:35.918153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b797ed-1d01-4646-969d-bf5f559bcfd8 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:35.919203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b797ed-1d01-4646-969d-bf5f559bcfd8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:35.919698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b797ed-1d01-4646-969d-bf5f559bcfd8 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:35.921321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 30/62] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:35.926755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a7a3-ba6f-4b55-925f-620a6bfee262 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:35.927169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a7a3-ba6f-4b55-925f-620a6bfee262 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:35.927314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21a7a3-ba6f-4b55-925f-620a6bfee262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:35.939549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27521-4a1f-4592-b7ab-4ab58e3d52a4 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:35.939834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27521-4a1f-4592-b7ab-4ab58e3d52a4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:35.940122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27521-4a1f-4592-b7ab-4ab58e3d52a4 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:35.941529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 31/63] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:35.946838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:35.947134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:35.947423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:35.950147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fc21a7a3-ba6f-4b55-925f-620a6bfee262 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:35.951367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fc21a7a3-ba6f-4b55-925f-620a6bfee262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:35.975999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:35.977592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:36.088317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 33/64] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:36.089004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.090789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396a4bc-9a02-4a1c-9c5d-859d86e6f6d6 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.091086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396a4bc-9a02-4a1c-9c5d-859d86e6f6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.091370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396a4bc-9a02-4a1c-9c5d-859d86e6f6d6 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.095688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 34/65] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:36.102081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.102548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.104206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.120110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 32/66] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:36.121696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.137319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:36.138716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:36.202964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.219320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 35/67] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:36.220383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.250656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9f150d-4507-49be-9bcb-e270fe9dae14 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.304052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.305161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.305595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.306020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.318303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:36.328980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:36.330045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:36.330931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:36.331500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773840a-04dd-4b10-b0ee-5b51e828ee27 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.333851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 33/68] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:36.334804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.366777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.367577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.367863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.368139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.380093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:36.387067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:36.389137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:36.390289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 36/69] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:36.391049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.444871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.455172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae4729-9114-4c77-8808-e27ad1affedc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.490356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.491394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.491676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.491955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.498106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a07f2-8a33-4ec6-b1cf-90e7c790a13a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.501874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a6f0e30a-58fc-4645-a696-dead7c3b9067 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:36.509878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:36.510699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:36.511494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 34/70] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:36.512399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff45f0-92f7-49c0-a1d6-a24c9bca7535 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.512674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff45f0-92f7-49c0-a1d6-a24c9bca7535 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.512941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff45f0-92f7-49c0-a1d6-a24c9bca7535 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.514332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 35/71] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:36.514934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff45f0-92f7-49c0-a1d6-a24c9bca7535 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.537157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.537826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.538115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.538395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.539176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:36.539487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:36.540945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e3d60-4f9d-4945-8564-60d93eb17949 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:36.549607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 37/72] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:36.550807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4bc2-cfd7-4905-a436-f9792c2629c3 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.551100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4bc2-cfd7-4905-a436-f9792c2629c3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.551388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c4bc2-cfd7-4905-a436-f9792c2629c3 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.552633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 38/73] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:36.553501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c4bc2-cfd7-4905-a436-f9792c2629c3 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.618684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.622533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff45f0-92f7-49c0-a1d6-a24c9bca7535 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.653845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c4bc2-cfd7-4905-a436-f9792c2629c3 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.655889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.656577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.656868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.657142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.657880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:36.658189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:36.659575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260591f2-938d-4ffc-97c0-fb4c8d04b717 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:36.664068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.668424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 36/74] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:36.669686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebfaed-10d1-4f91-bcc4-11336a7c0900 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.669977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebfaed-10d1-4f91-bcc4-11336a7c0900 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.670255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebfaed-10d1-4f91-bcc4-11336a7c0900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.686737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8bebfaed-10d1-4f91-bcc4-11336a7c0900 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:36.687660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.687908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8bebfaed-10d1-4f91-bcc4-11336a7c0900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:36.688352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.688646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.688928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.689678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:36.689998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:36.691403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:36.699682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 39/75] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:36.700494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.758642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 37/76] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:36.759974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.760270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.760581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.778531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:36.779741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:36.809351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef381884-c17d-4c23-b6e0-5482fb587bbb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.844074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.844977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.845264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.845541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.846107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 38/77] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:36.846927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.862289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:36.864006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:36.869938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 40/78] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:36.869938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:36.873347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.898060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:36.947204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c992fa9a-f176-4357-b61c-a328a6935c12 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.967456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46a70a-81d0-4ca8-ad83-39887a6a4fc6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:36.988810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:36.989692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:36.989976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:36.990253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:36.991465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:36.992991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:36.994631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 39/79] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:36.996087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.006035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.006944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.007232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.007512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.008801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.010223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.019701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 41/80] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:37.019701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.097006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.111686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573f6fa-df9d-4683-8b57-3d66e652a3d1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.116392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.120967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d4109-4d24-47ec-9ed3-561814339e8e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.146295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.147097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.147385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.147668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.155955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.156790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.157102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.157379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.157834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:37.158529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.160002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.161644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 42/81] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:37.162397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:37.165534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:37.166407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:37.167167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 40/82] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:37.172014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.259843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d86118-8eec-4444-9742-0081565ff10b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.271597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c645c07-ddc0-4d03-8910-45f6b10284ce None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.293382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.294022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.294305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.294579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.303402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.304042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.304350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.304635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.322819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.328932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.364600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.366090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.372494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.373964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.463941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 43/83] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:37.464827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.474328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 41/84] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:37.475238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.528721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.530800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.565475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db88004-2d77-4657-ade3-fba23f74abae None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.577192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7356b8-0821-48f5-8be1-2b7e42334cc6 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.597799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.598446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.598724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.598995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.610545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.611211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.611492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.611764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.612496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.613948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.643498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 42/85] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:37 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:37.644342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.680950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.682497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.754005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f526b-66c1-4c9b-8c2c-0ebadb6caa09 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.754212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.765782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:37.800050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 44/86] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:37.800050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.802621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.803492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.803913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.804376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.805294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:37.805609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.807090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.820345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 43/87] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:37.820345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.934120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1265830-b69a-4a21-88d8-2235c9a26953 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.940131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe4063-d901-404e-b191-091ee94f72a8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:37.976029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.976992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.977299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.977541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.978437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e454-837c-488b-b65b-981305109a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:37.978810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.979301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e454-837c-488b-b65b-981305109a88 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:37.979898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e454-837c-488b-b65b-981305109a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:37.980712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745e454-837c-488b-b65b-981305109a88 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:37.981017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.981954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745e454-837c-488b-b65b-981305109a88 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:37.982960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 45/88] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:37.984436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745e454-837c-488b-b65b-981305109a88 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:37.984560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:37.985534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 44/89] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:37.986237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e454-837c-488b-b65b-981305109a88 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.024037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.034183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.112832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70489171-737e-4d5c-9743-2afb68abcd99 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.119248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745e454-837c-488b-b65b-981305109a88 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.160200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.161112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.161240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.161388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.161669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.162411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.162759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.162866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.163125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.164336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.164439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.165959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.166079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 46/90] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.166791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.172609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 45/91] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:37 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.173561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.286995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a6e60a-06dd-42a1-b38d-c1a1ed4ad682 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.291023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.296121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c48b-24e5-4746-83eb-348433511931 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.300623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.327503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.328089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.328401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.328680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.329517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=3a944101ee294a37aa42cee1133a9032) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:38.329830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.331281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.337095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 46/92] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/role_assignments?user.id=3a944101ee294a37aa42cee1133a9032&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.338087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.340883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.341958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.343327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.343744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.344762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:38.345189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.346834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.365133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 47/93] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.366588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.515153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e4e41-eea8-4d5e-ad92-0a7195c5d7cc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.554384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.555060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.555347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.555628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.621231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.639430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.641006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.675146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f58e1f-54ad-4575-8b80-91756d896aab None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.756485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 48/94] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:38.763322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.798848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.799578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.799859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.800130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.801018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:38.801331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.802794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.819265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 47/95] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 259 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.836673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:38.873545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950ca3-ef00-4b3f-bb93-659ea9bcacd0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:38.912689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:38.913589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:38.913875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:38.914151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] PATH_INFO: `/v3/projects/512e614fbe7848d9a4a422428b3d6e74` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:38.915492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=512e614fbe7848d9a4a422428b3d6e74)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:38.917569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f1168-14db-40d1-8c0d-39b01ef8c5b4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:38.919323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 49/96] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/projects/512e614fbe7848d9a4a422428b3d6e74 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:38.925108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:38.960807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:39.034796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c10b4-5443-4add-afcc-14293fa71685 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:39.077908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:39.078312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:39.078598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:39.078881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:39.079734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=512e614fbe7848d9a4a422428b3d6e74, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:39.080052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:39.081583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170bbe-a5f7-4e0e-bed9-d3dc898349f1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:39.087982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 48/97] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:38 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=512e614fbe7848d9a4a422428b3d6e74&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:40.224482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a026d97-8fcf-4119-97f3-cded11bbaf53 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:40.224987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a026d97-8fcf-4119-97f3-cded11bbaf53 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:40.225059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a026d97-8fcf-4119-97f3-cded11bbaf53 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:40.226402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 50/98] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:40.232486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:40.232892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:40.233058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:40.253257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:40.254459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:40.333004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 49/99] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:40.368485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a026d97-8fcf-4119-97f3-cded11bbaf53 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:40.494717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a026d97-8fcf-4119-97f3-cded11bbaf53 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:40.530913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:40.531717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:40.532002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:40.532280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:40.534244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:40.543715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:40.551949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:40.552883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:40.553871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 51/100] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:40.560935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:40.595447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:40.694123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eadd90-cab0-4a6e-81bf-f656d5690b10 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:40.737920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:40.738651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:40.738942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:40.739225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:40.739985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:40.740311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:40.741714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:40.750988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 50/101] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:40.756524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:40.882179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987a232-ee4f-45f5-8301-005b61a586f9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:40.924384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:40.925276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:40.925563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:40.925843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] PATH_INFO: `/v3/users/020d6fac177149c9ae9ee59b1300dfb8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:40.953056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] RBAC: Authorizing `identity:get_user(user_id=020d6fac177149c9ae9ee59b1300dfb8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:40.954563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:40.956324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 52/102] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:40 2022] GET /identity/v3/users/020d6fac177149c9ae9ee59b1300dfb8 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:40.961817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:41.061595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:41.078011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97d7ba9-c276-41df-ae76-f3e92e39cba5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:41.131839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:41.132584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.132820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.133100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.216126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:41.217646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebabba6-dfd3-4d1e-a411-e1dabf7d27c7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:41.320750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:41.329851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 51/103] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:41.335343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:41.345668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733d1-e83d-42c9-8b13-55972668f2ea None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.345998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733d1-e83d-42c9-8b13-55972668f2ea None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.346306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc733d1-e83d-42c9-8b13-55972668f2ea None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.347724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 52/104] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:41.352980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9862c5-6947-4cf5-8a15-40a85da1e9e0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.353305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9862c5-6947-4cf5-8a15-40a85da1e9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.353618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9862c5-6947-4cf5-8a15-40a85da1e9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.389389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8a9862c5-6947-4cf5-8a15-40a85da1e9e0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:41.390869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8a9862c5-6947-4cf5-8a15-40a85da1e9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:41.416746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:41.461306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b560751-65a7-496f-b679-4c9ca3d7349e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:41.516322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 53/105] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:41.517857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5ad6-ccb7-4f6c-badf-d8cf20aaef4d None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.518563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5ad6-ccb7-4f6c-badf-d8cf20aaef4d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.518985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b5ad6-ccb7-4f6c-badf-d8cf20aaef4d None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.520585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 54/106] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:41.526085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.527085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.527504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.543554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:41.544737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.545163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.545604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.548292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:41.549988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:41.551873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 53/107] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:41 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:41.556516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:41.558025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:41.561712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:41.712512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 55/108] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:41.713581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:41.767038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b7a24-d301-44db-8ca0-23390ff5ead7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:41.788569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:41.793664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:41.822385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:41.823949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.824426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.824843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.847221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:41.855426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:41.856455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:41.857382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:41.872198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 54/109] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:41.878028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:41.899718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970cb515-b7b4-4390-a178-15e9134c0d77 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:41.963145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:41.964205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:41.964707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:41.965178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:41.966236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=020d6fac177149c9ae9ee59b1300dfb8) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:41.966769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:41.968456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:41.981334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 56/110] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:41 2022] GET /identity/v3/role_assignments?user.id=020d6fac177149c9ae9ee59b1300dfb8&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:41.984660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.117791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba99f87-b3b7-417b-86b4-d98c4339864c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.205646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.206573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.206925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.207291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.208988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:42.210680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f8aef-d21c-4dc6-a7ac-2846ba1655d1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:42.212559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 55/111] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:42.220404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5825e-07c4-4d07-a15a-6c086c681270 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.220558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f36b00-16a7-42e1-b14c-2edf1bac0d38 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.221134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5825e-07c4-4d07-a15a-6c086c681270 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.221621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5825e-07c4-4d07-a15a-6c086c681270 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.223233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 56/112] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:42 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:42.224155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5825e-07c4-4d07-a15a-6c086c681270 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.232272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.256196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.257266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.257806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.258290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.259249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:42.259747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:42.262161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec14822-34c4-4af9-bf78-6e0265eb6317 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:42.272051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 57/113] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:42.273730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeca02-afb7-4328-b751-5ab065eb799a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.274221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeca02-afb7-4328-b751-5ab065eb799a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.274722 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbeca02-afb7-4328-b751-5ab065eb799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.293237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-adbeca02-afb7-4328-b751-5ab065eb799a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:42.294865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-adbeca02-afb7-4328-b751-5ab065eb799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:42.304903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.359829 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc5825e-07c4-4d07-a15a-6c086c681270 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.373212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 58/114] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:42.374677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5716b3-d086-4792-aac0-2c0231754203 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.374988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5716b3-d086-4792-aac0-2c0231754203 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.375244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5716b3-d086-4792-aac0-2c0231754203 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.376686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 59/115] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:42.377289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5716b3-d086-4792-aac0-2c0231754203 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.420126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.420823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.421105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.421378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.422594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:42.424545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16affe5-fc26-4587-aacd-dd335920cc9e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:42.451998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 57/116] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:05:42 2022] POST /identity/v3/projects => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:42.453185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d451e00-d747-4d8a-adb0-e64bead4a34f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.453500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d451e00-d747-4d8a-adb0-e64bead4a34f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.453760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d451e00-d747-4d8a-adb0-e64bead4a34f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.455508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 58/117] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:42.460966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d0ee9-6b8e-4961-b15f-cbff86a2c13f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.461258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d0ee9-6b8e-4961-b15f-cbff86a2c13f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.461529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d0ee9-6b8e-4961-b15f-cbff86a2c13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.482851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-634d0ee9-6b8e-4961-b15f-cbff86a2c13f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:42.484155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-634d0ee9-6b8e-4961-b15f-cbff86a2c13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:42.501617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5716b3-d086-4792-aac0-2c0231754203 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.504706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.543563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.544511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.544794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.545069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.546357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:42.547976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:42.549930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 60/118] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:42.550533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.589230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 59/119] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:42.591045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.591545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.592015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.614577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:42.616158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:42.650652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.698946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c149-8a83-4ed4-b7bf-51b3f3ba4a24 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.711729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 60/120] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:42.715079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.760571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.761395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.761683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.761959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.776944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:42.788143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:42.789045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:42.789802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 61/121] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:42.790913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.829807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31891e54-7ded-4ebf-b58b-e0f60fe18f8b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.835962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.871690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.871690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.871690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.871690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.872562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:42.874719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:42.877588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 61/122] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:42.878437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:42.913077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:42.926509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785d9ea-97b0-4427-86c4-89b3febd6199 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:42.970132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:42.971186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:42.971619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:42.972044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:42.983957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:42.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:42.993243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:42.994011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:42.996395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 62/123] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:42.997528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.004518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6e791-047f-44f2-bcc0-8e0f83020f22 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.051887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.053014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.053303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.053585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.067244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:43.076502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:43.077385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 62/124] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:43.080076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.114879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.125355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a2bf-ca5f-4782-bb90-9861b0716ac9 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.164119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.165215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.165693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.166174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.167682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:43.168200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.169848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.178737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 63/125] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.179909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.202528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.217325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a0d56-f18f-4b4b-bd5d-c72cc19930e7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.264857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.266407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.267013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.267322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/users/3a944101ee294a37aa42cee1133a9032/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.270471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, user_id=3a944101ee294a37aa42cee1133a9032, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.271958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.289596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 63/126] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:43 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/users/3a944101ee294a37aa42cee1133a9032/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:43.290557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.318899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3a572-bf52-4c04-9721-dd3c9bc1a456 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.385159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.386445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.386704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.388800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.388800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:43.388800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.389496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.413686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 64/127] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.418685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.425712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781ae17-88a4-4be6-8e09-5eb0b04c2d83 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.458256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.518458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.519284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.519512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.519813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.520647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:43.520981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.522393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.540222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 64/128] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.541269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.558200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.562318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd36f50-44d0-4b7b-9fd6-c8288fdf4a34 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.602149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.603361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.603853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.604924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.606436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.608103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.610268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 65/129] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.611250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.671141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65506939-d63b-4fe6-bc69-c37e1d85b453 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.707850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.709143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.709680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.710183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.712003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.714388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.716509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 65/130] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.717290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.717858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.745277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a16cc4-75c6-4c87-9c22-f78235d72acf None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.807633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.809079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.812206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.812713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.814213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.816108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.819136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 66/131] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.821919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:43.871938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:43.919132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dac74b-e2d4-47b8-9d91-787496b572c5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:43.962069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:43.963694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:43.964745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:43.965236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:43.967011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:43.968751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:43.970976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 66/132] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:43.972010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.003527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acad7-3460-4f1e-91b1-9babc0afaf5f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.041412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.043358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.043843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.044354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.045832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:44.047516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:44.049468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 67/133] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:44.050431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.074501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:44.155878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597add2-b6a0-4be2-9820-d1dd69d236ac None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.190874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.192359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.192851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.193346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.195729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:44.197471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:44.220544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 68/134] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:44 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:44.227291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.239730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22755f3-2cec-4bf9-a24e-b6b5135c8b0f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.336257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.337621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.338131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.340827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] PATH_INFO: `/v3/projects/512e614fbe7848d9a4a422428b3d6e74` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.342489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] RBAC: Authorizing `identity:get_project(project_id=512e614fbe7848d9a4a422428b3d6e74)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:44.345355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:44.348371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 67/135] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:43 2022] GET /identity/v3/projects/512e614fbe7848d9a4a422428b3d6e74 => generated 312 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:44.357575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.381302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:44.463590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:44.511521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c6efd-59f9-43b0-bb2a-4bc7908e5b9d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.527869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948811da-949e-4c0f-a8c5-981f40affd93 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.556957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.557920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.558216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.558477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] PATH_INFO: `/v3/projects/512e614fbe7848d9a4a422428b3d6e74/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.561352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] RBAC: Authorizing `identity:create_grant(project_id=512e614fbe7848d9a4a422428b3d6e74, user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:44.562829 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ee871-f975-49a2-b427-6fd816708301 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:44.571348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.572471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.572810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.573114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.575737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243, role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:44.577369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:44.578405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 68/136] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:44 2022] PUT /identity/v3/projects/512e614fbe7848d9a4a422428b3d6e74/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:44.579976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a7e0-6804-4bc6-9b6c-8a59672bc952 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.580269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a7e0-6804-4bc6-9b6c-8a59672bc952 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.580569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18a7e0-6804-4bc6-9b6c-8a59672bc952 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.582111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 69/137] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:44.590476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.590808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.591038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.594757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 69/138] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:44 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/users/42b1dfbd9a6b4e23b6dc61a02d6ea243/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:44.611523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:44.612834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:44.695593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 70/139] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:44.719617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.861842 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7ecb3-91db-4c1f-b6f5-03e36335c808 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:44.899465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:44.900280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:44.900600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:44.900874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:44.912533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:44.916078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:44.925867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:44.926765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:44.927507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 70/140] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:44.933171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:44.936409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:45.096604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb864f3-543a-48f1-b754-484de0004fc6 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:45.137968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:45.138659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.138945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.139224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.139984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:45.140291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:45.141746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:45.150672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 71/141] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:45.156858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:45.293267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b0426-57d3-4ee1-9e7a-0ce9625b0272 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:45.328495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:45.329313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.329597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.329872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] PATH_INFO: `/v3/users/020d6fac177149c9ae9ee59b1300dfb8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.331072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] RBAC: Authorizing `identity:get_user(user_id=020d6fac177149c9ae9ee59b1300dfb8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:45.332562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30af0d-e685-4784-9e90-86c0463e2081 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:45.334624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 71/142] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:45 2022] GET /identity/v3/users/020d6fac177149c9ae9ee59b1300dfb8 => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:45.339841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:45.400913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7e937-c86a-4259-8491-9fbea6bed0d0 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.401895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7e937-c86a-4259-8491-9fbea6bed0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.402281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7e937-c86a-4259-8491-9fbea6bed0d0 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.403758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 72/143] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:45.408902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.409309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.409693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.416666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:45.438726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:45.439936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:45.453884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c385e-d00c-453b-af4f-beaceba2e833 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:45.490875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:45.491750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.492042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.492333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.494352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:45.497500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:45.500673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 72/144] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:45 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:45.506446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:45.562433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:45.574428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 73/145] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:45.622892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:45.659779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e6d14-5fa5-402c-9ddf-c4da5f3edd3b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:45.723206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:45.724511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.725110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.725532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/users/020d6fac177149c9ae9ee59b1300dfb8/roles/c81cae8588bc4b589a64a00b7d77bffb` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.730502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, user_id=020d6fac177149c9ae9ee59b1300dfb8, role_id=c81cae8588bc4b589a64a00b7d77bffb)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:45.732247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:45.734994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:45.737156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ebfa6-4429-43ea-b221-bb400678f0d5 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:45.765605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 73/146] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:45 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/users/020d6fac177149c9ae9ee59b1300dfb8/roles/c81cae8588bc4b589a64a00b7d77bffb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:45.816813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:45.817627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:45.817910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:45.818185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:45.821254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:45.822869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a72da1-956a-41ca-b2ca-38fbbc81d950 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:45.825122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 74/147] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:45.830441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:45.972580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed920e-d39f-492e-84ad-ca1ffe647dca None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:46.021011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:46.021694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:46.021979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:46.022254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:46.022982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:46.024478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a95827-7232-4209-bbfa-b5f8f55715fe None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:46.055930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 74/148] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:45 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:46.508185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134665-b385-47f9-8720-d3f6c77b9925 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:46.508928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134665-b385-47f9-8720-d3f6c77b9925 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:46.509410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7134665-b385-47f9-8720-d3f6c77b9925 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:46.511107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 75/149] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:46.516401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:46.516948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:46.517424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:46.545345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:46.546863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:46.605885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:46.630458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 75/150] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:46.656536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134665-b385-47f9-8720-d3f6c77b9925 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:46.738679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:46.789987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7134665-b385-47f9-8720-d3f6c77b9925 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:46.831847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:46.832676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:46.832956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:46.833235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:46.849801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:46.858111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:46.858966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:46.860163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 76/151] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:46.865477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.021276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634297d-47b7-4d97-8662-bd2dc4a41b49 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:47.089627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:47.090321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.090598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.090863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.091630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:47.091932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:47.093382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:47.102592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 76/152] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:47.108156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.257717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8977f97a-f373-4af1-b6ad-bc130f99a36f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:47.290804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:47.303962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:47.305443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.306119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.306646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] PATH_INFO: `/v3/users/3a944101ee294a37aa42cee1133a9032` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.307931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] RBAC: Authorizing `identity:get_user(user_id=3a944101ee294a37aa42cee1133a9032)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:47.309802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b792239-9d90-4992-8ec9-d8162a13b13a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:47.311633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 77/153] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:47 2022] GET /identity/v3/users/3a944101ee294a37aa42cee1133a9032 => generated 247 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:47.316969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.408341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:47.471943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060aa5c-d6fa-4598-9457-3c11ac340adc None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.472254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060aa5c-d6fa-4598-9457-3c11ac340adc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.472546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060aa5c-d6fa-4598-9457-3c11ac340adc None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.474275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 78/154] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:47.479743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.480715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.481475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.486138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3134d-2ba8-4630-ac6a-76d21628303a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:47.512728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:47.514241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:47.565114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:47.566368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.567775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.569104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.629497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 79/155] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:47.670420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.705564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:47.707121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:47.772100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:47.796280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8e810-7c40-466e-9666-763ff4eff503 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:47.824411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:47.839644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:47.840866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.841340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.841832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.844527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 77/156] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:47.850828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.860019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:47.873409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:47.874501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:47.875330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:47.877935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 80/157] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:47.885647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:47.955094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cbe9fa-95e7-4d1c-9d0a-8ce2d43bf760 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:47.990449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:47.991307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:47.991585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:47.991855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:47.998518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.000056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33120d4-54f5-4b9f-bb14-c936800ae660 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.001795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 78/158] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:47 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:48.003265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296d976-3f12-456b-bbbd-1cc0c199cedc None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.003265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296d976-3f12-456b-bbbd-1cc0c199cedc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.003554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296d976-3f12-456b-bbbd-1cc0c199cedc None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.004804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 79/159] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:48.011111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296d976-3f12-456b-bbbd-1cc0c199cedc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.023612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8837-2a1a-40ce-8ed6-4c847fb28e7d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.029990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.072396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.073393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.073883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.074348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.080393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:48.080950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.082582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c2707-ef10-4eaf-b8e6-0bc14012009b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.092339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 81/160] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:48.094153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.094634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.095125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.114591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:48.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:48.163560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.180988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296d976-3f12-456b-bbbd-1cc0c199cedc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.193476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 82/161] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:48.194741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.235306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.235984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.236263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.236557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.237412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=3a944101ee294a37aa42cee1133a9032) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:48.237736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.239204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.245802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 80/162] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/role_assignments?user.id=3a944101ee294a37aa42cee1133a9032&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:48.246542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.313851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26125b0f-19ad-427e-8b0f-8271c5e01d17 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.353044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.354375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.355344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.355933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.357175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.358982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.361357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.363493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 83/163] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:48.369767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.400784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0db24-cf7e-4f40-b51b-2b9a39554a92 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.422768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.470036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.470897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.471174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.471438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.489023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3a178-b82a-4882-9808-1d3bece8c0f2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.497197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:48.505472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:48.506384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:48.507586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 81/164] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:48.513098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.532616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.533628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.534205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.534671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.613023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.614896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.662482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c17d-3857-415f-9607-dc1156603d60 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.665249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.718114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.718796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.719077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.719349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.720108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:48.720433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.721847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.723402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.730000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 84/165] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:48.736098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.737181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 82/166] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:48.742366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.867274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfdcc69-e214-4eb8-94e8-00842a86cd6c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.879893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae9616-ddff-413c-9ade-508c5da95563 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:48.915576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.916532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.917310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.917586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.925054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.925836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:48.926541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2cb273-e5ce-4ad2-b69a-24acc9a608bb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.927160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.927634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.928113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.928542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 83/167] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:48.929773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a37d6-dfa6-48e5-9248-daaef00c3ad0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.930055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a37d6-dfa6-48e5-9248-daaef00c3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.930269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:48.930373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a37d6-dfa6-48e5-9248-daaef00c3ad0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.931531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 84/168] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:48.932424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:48.934889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 85/169] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:48.939669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:48.939669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:48.939669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:48.941248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:48.961750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:48.973310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:48.975737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:49.059425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e908328-d8d0-43d4-a487-37b65628bf88 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.060863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:49.069350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 85/170] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:49.070179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.119116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.120675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.121180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.121646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.195896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:49.197744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:49.213465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:49.214919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b481f5-58ea-4e49-be35-cf21500b01a4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.277553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.277919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.278145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.278413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.279259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:49.279566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:49.281059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:49.287449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 86/171] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:49.288255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.314358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 86/172] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:49.325528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.384679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:49.448757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3c09aa-229b-4b03-bee0-707231cd2c42 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.513090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.513931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.514270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.514550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.531256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:49.547568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:49.548627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:49.549828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 87/173] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:49.554811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.620250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedfc8f-8a11-493f-a5ed-168f8f105e12 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.646418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:49.700810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34f8e00-2d2f-4cce-9fc7-0f56f3eff6bb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.705575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.706471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.706744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.707020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.713137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:49.714681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:49.716784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 87/174] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:49.725253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.731554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:49.752997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.753719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.753953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.754228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.754982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:49.755290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:49.756780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:49.765994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 88/175] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:49.771545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:49.900207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e3ac8-f5ed-4882-b1a2-11658c58672f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.963847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:49.964748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.965029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.965303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.966142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:49.966454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:49.967905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49812-5d53-4ee7-8a97-6157c2f081ea None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:49.971439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556eb9c-64b1-412e-9cf6-c8454de4a506 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:49.982683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 88/176] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:49.983965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22ba7-5e96-4ff5-a414-4355ec4d2b08 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.984255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22ba7-5e96-4ff5-a414-4355ec4d2b08 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.984561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22ba7-5e96-4ff5-a414-4355ec4d2b08 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:49.987985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 89/177] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:49.994080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:49.994368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:49.994646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.026485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:50.027710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:50.037502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.038437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.038719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.038993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] PATH_INFO: `/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.040529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] RBAC: Authorizing `identity:get_user(user_id=42b1dfbd9a6b4e23b6dc61a02d6ea243)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:50.044257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:50.048808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 89/178] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:49 2022] GET /identity/v3/users/42b1dfbd9a6b4e23b6dc61a02d6ea243 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:50.054149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.117569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:50.157831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:50.161946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 90/179] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:50.197449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.279264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f98769-abf1-4fa9-83bc-68c12603cd8f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:50.358451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.359128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.359421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.359702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.426584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921a86b-4b8d-48cc-a7ec-22a69100fd57 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:50.457837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:50.459559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:50.497028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.497841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.498124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.498398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.519708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:50.520647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:50.527987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:50.529217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:50.530124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 91/180] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:50.537228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.591574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 90/181] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:50.597664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.640690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:50.784604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13398965-bfd6-4bfd-8652-f5e4bddcffd5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:50.785308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a59b3c-606b-41b1-b8c1-836361434713 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:50.832743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.833606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.833883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.834157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] PATH_INFO: `/v3/projects/512e614fbe7848d9a4a422428b3d6e74` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.835512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] RBAC: Authorizing `identity:get_project(project_id=512e614fbe7848d9a4a422428b3d6e74)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:50.837003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:50.841662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 91/182] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/projects/512e614fbe7848d9a4a422428b3d6e74 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:50.846181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.859249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.859946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.860230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.860551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.861310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:50.861620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:50.863026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:50.866319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:50.891926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 92/183] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:50.897032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:50.955474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b4fc9-79ef-4706-9340-a3b87e0eebe4 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:50.994300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:50.995533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:50.995814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:50.996085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:50.996988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=512e614fbe7848d9a4a422428b3d6e74, user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:50.997303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:50.998836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b36552-942a-44f5-a16f-12ce378297e5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.005860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 92/184] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/role_assignments?user.id=42b1dfbd9a6b4e23b6dc61a02d6ea243&scope.project.id=512e614fbe7848d9a4a422428b3d6e74&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:51.007258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d3c9f-5738-46fe-a5fd-a00b8fb35cf4 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.007551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d3c9f-5738-46fe-a5fd-a00b8fb35cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.007826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d3c9f-5738-46fe-a5fd-a00b8fb35cf4 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.009547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 93/185] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:51.015511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.015887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.016838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.042582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:51.044226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:51.062367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.071015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d188207-8ad8-4f1e-956a-2813daa130f0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.108849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.109888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.110270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.110639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] PATH_INFO: `/v3/users/020d6fac177149c9ae9ee59b1300dfb8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.111976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] RBAC: Authorizing `identity:get_user(user_id=020d6fac177149c9ae9ee59b1300dfb8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.113638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.115461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 93/186] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:50 2022] GET /identity/v3/users/020d6fac177149c9ae9ee59b1300dfb8 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:51.125495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.151171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 94/187] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:51.186537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.204883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.255803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc1edf-3fd9-43f4-b941-a5cedfca0f07 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.301199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.302016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.302304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.302602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.340147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.343832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f5e5e-c997-40f1-81f0-b2359fa536e4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.392064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.394317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba3221-1fa3-4f69-a1a0-e4939da87dfa None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.395153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.397284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.397767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.398218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.412071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-fcd823b2-1e22-42ac-8ee0-d80f92c2019d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:51.421179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:51.422165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:51.422943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 95/188] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:51.424192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b830f-cc0a-40cf-8c29-668bcdef22af None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.424502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b830f-cc0a-40cf-8c29-668bcdef22af None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.424775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2b830f-cc0a-40cf-8c29-668bcdef22af None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.426127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 96/189] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:51.430960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b830f-cc0a-40cf-8c29-668bcdef22af None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.501234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.511193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 94/190] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:51.513692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.513981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.514279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.529974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:51.531127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:51.533772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2b830f-cc0a-40cf-8c29-668bcdef22af None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.566888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.567488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.567770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.568046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.568830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:51.569137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.570522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692dad7f-1a4b-44d0-9b2a-b5f4ee753bdb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.579540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 97/191] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:51.580850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d875-4797-4877-b677-7ef06ab5b2e0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.581135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d875-4797-4877-b677-7ef06ab5b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.581405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85d875-4797-4877-b677-7ef06ab5b2e0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.584336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 98/192] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:51.584336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85d875-4797-4877-b677-7ef06ab5b2e0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.590843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.599652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 95/193] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:51.600353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.693194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85d875-4797-4877-b677-7ef06ab5b2e0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.697254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb07002-f3e0-49c6-9e69-49617b528fc6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.722046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.727269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.728109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.728419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.728691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.730612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.731558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.732075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf28f1-34d2-4d08-80ad-65c26ef9fdef None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.732464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.732745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.733016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.733784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 99/194] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:51.734974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.735256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.735525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.748441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.749908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.751593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 96/195] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:51.751780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:51.752385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.753058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:51.808255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.832358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 100/196] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:51.832358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.867438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7abefc-03b4-4889-a4f7-13a1300a1037 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.900999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.901811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.902098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.902377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.912480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:51.920201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512c1da-1959-4495-b314-8eddd26a0a73 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:51.921101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:51.922002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:51.922763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 97/197] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:51.923441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:51.954237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:51.960054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:51.960736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:51.961016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:51.961283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:51.962105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, scope.project.id=81e587db604e403584fcdc9ec139396c, user.id=020d6fac177149c9ae9ee59b1300dfb8) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:51.962407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:51.964405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:51.970144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 101/198] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/role_assignments?user.id=020d6fac177149c9ae9ee59b1300dfb8&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:51.970918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.035719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa2483-4d74-4f65-9275-ad876a88db3c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.051666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.082185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.082882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.083055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.083312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.088422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3842a1f-c390-446e-9498-f096e63b01c7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.128138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.128974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.129260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.129528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.130762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.132336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.134184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 102/199] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:52.134810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.165906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.168002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.212325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.256152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd7100-994b-4ec7-a8ca-8cadbfbe8f6a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.276647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 98/200] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:52.277466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.293313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.295405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.296231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.296550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.296819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.297573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:52.297857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.299240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.308454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 103/201] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:52.309897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.378490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363645-1d62-4377-8990-afeed75e2aad None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.411562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.412335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.412618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.412893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.413591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.415043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.415136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43a5b8-7529-4178-8322-03da7b2007e6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.434754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 99/202] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:52 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:52.437260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.440005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.453821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.454708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.454999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.455264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.456599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.458107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.460054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 104/203] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:52.469445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.517826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.560209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442d96a-891b-49c7-b7a0-3fd6d788c381 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.586515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a756559-bfe3-43f8-b14e-b2ab3f3c9bfb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.598961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.599849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.600125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.600432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.601681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.603173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.605389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 100/204] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:52.610665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:52.631303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.631882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.632161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.632458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.633458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=99d6a6d18f3849f1af29c47b794d01a9) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:52.633633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.635085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.641414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 105/205] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/role_assignments?user.id=99d6a6d18f3849f1af29c47b794d01a9&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:52.738816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81af78-f5c3-415e-b3e4-3ed7f27eb81d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:52.779798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:52.780760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:52.781185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:52.781600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:52.786062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:52.857805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:52.859523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:52.965268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 101/206] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:05:52.971055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:53.032680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:53.078529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd5c8-c1fd-4988-a1e6-4a0f158a6a67 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:53.113903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:53.114812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:53.115089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:53.115409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:53.116619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:53.118112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e65e9e3-5542-4a0c-807e-21cf63a94a55 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:53.120322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 106/207] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:52 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:53.126238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:53.162007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:53.230686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea84f-1dc5-4e5f-8aba-222af01ad8e4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:53.266046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25197528-d040-405c-a0c2-03caddb563de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:53.266703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197528-d040-405c-a0c2-03caddb563de None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:53.266983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197528-d040-405c-a0c2-03caddb563de None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:53.267256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25197528-d040-405c-a0c2-03caddb563de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:53.268081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25197528-d040-405c-a0c2-03caddb563de None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=99d6a6d18f3849f1af29c47b794d01a9) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:53.268416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25197528-d040-405c-a0c2-03caddb563de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:53.269850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25197528-d040-405c-a0c2-03caddb563de None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:53.275858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 102/208] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:53 2022] GET /identity/v3/role_assignments?user.id=99d6a6d18f3849f1af29c47b794d01a9&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:54.821563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d1bf-3fbf-43db-9f92-0f567aeca8de None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.822073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d1bf-3fbf-43db-9f92-0f567aeca8de None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.822147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d1bf-3fbf-43db-9f92-0f567aeca8de None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.823526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 107/209] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:54.829061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164883d9-be6d-4866-a0fd-d50675a4fdec None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.829481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164883d9-be6d-4866-a0fd-d50675a4fdec None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.829650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164883d9-be6d-4866-a0fd-d50675a4fdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.846805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-164883d9-be6d-4866-a0fd-d50675a4fdec None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:54.848067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-164883d9-be6d-4866-a0fd-d50675a4fdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:54.872486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da9cd-3470-4762-b648-44c2f147e7b1 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.873129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da9cd-3470-4762-b648-44c2f147e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.873607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49da9cd-3470-4762-b648-44c2f147e7b1 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.875172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 108/210] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:54.880214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a55-d0e4-44f3-9623-898152e2a3e2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.880757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a55-d0e4-44f3-9623-898152e2a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.881251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8351a55-d0e4-44f3-9623-898152e2a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.898260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e8351a55-d0e4-44f3-9623-898152e2a3e2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:54.899752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e8351a55-d0e4-44f3-9623-898152e2a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:54.925298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 103/211] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:54.947345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fd4f4-ad84-4c4e-922b-d849a16e0c3a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.947680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fd4f4-ad84-4c4e-922b-d849a16e0c3a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.947979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fd4f4-ad84-4c4e-922b-d849a16e0c3a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.950039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 104/212] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:54.953554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fd4f4-ad84-4c4e-922b-d849a16e0c3a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:54.973308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 109/213] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:05:54.974592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:54.974907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:54.975181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:54.989903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:54.999558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:55.001050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:55.060375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.072438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fd4f4-ad84-4c4e-922b-d849a16e0c3a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.086788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 110/214] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:55.089129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.115251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.116053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.116357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.116635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.127170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:55.134909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:55.135798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:55.136909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 105/215] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:05:55.137866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.201375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d93b-a092-483b-8a4e-63ea72e7c9be None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.235757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.238417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.239206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.239483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.239749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.240917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.242415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.244049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 111/216] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:55.244837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.252423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e0270-aa16-461b-a242-659d36dfd0ca None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.288217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.289080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.289360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.289632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.303875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:55.308535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:55.312317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:55.314063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 106/217] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:55.314838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.366389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4ccf8-decb-4690-bf22-f6b62dcb5516 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.400233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.401507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.401789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.402057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.402792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:55.403101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.404503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.412801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 112/218] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:55.417936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.423158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2501833a-6d31-4372-ba09-9115d79a1167 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.460658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.461593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.462023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.462569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.464108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.465601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.486690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 107/219] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:55 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:55.487426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.496673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.532993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa6fe-fb6d-446f-9eb3-f16521b20b49 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.548842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.572643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.573992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.574279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.574555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.575327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:55.575632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.577210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033eb78-7945-4fca-b159-af2f75a5af51 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.587182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 113/220] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:55.589076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00b684-8440-47d6-8219-403397629fc4 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.589368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00b684-8440-47d6-8219-403397629fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.589653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb00b684-8440-47d6-8219-403397629fc4 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.595252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 114/221] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:55.596188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00b684-8440-47d6-8219-403397629fc4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.600585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9bc53c-4c63-47c9-957e-b8a2329289a9 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.636179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.637014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.637293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.637570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] PATH_INFO: `/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.652005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] RBAC: Authorizing `identity:get_user(user_id=76c9c4de896d4bc5a6e4f5bb08d20721)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.653535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a0c3c-5789-46cd-95f6-7753a543b3eb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.655305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 108/222] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:55.656592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.656876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.657149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.674196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:55.675399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:55.718769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb00b684-8440-47d6-8219-403397629fc4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.748030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.759438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.759626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 109/223] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:55.760573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.761168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.761582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.761992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.763362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.765108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.767228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 115/224] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:55.777944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:55.848386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:55.892157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727de25-fa65-4565-8767-d92b26e32b5d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.920433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be039b-8f3f-46d3-985d-b5ac5fa50bd1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:55.928482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.929274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.929545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.929813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.967250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:55.968176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:55.968753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:55.969034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:55.971199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:55.972736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:55.975344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 116/225] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:55.976139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.004043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.013043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.014802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.109825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb21e20-6260-4687-90ca-3dd3ef02da55 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.124698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.140655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 110/226] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:56.141577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.149253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.150195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.150487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.150763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.163497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:56.172324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:56.173385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:56.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 117/227] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:05:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:56.174968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.245474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.254157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a446dec-139e-44dd-b682-af8b64a9c675 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.291074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.292166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.292590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.292984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/users/99d6a6d18f3849f1af29c47b794d01a9/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.295076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, user_id=99d6a6d18f3849f1af29c47b794d01a9, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.296680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.303557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ca0ca-eb58-480f-b2c8-bc003206e38b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.312784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 111/228] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:56 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/users/99d6a6d18f3849f1af29c47b794d01a9/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:56.314049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.345955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.346782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.347068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.347342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.348742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.350506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.352869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 118/229] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:56 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:56.358571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.457183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.501518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc656b-99fc-434c-91ae-c1d9befa838f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.570351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36585310-bca9-4a81-8a76-c177efcf2dd2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.570806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.571489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.571771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.572045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.573467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:56.573782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.575159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.583893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 112/230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:05:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:56.590094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.610114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.610919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.611202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.611475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.612338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=76c9c4de896d4bc5a6e4f5bb08d20721) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:56.612642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.613380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.614778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.626932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 119/231] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:56 2022] GET /identity/v3/role_assignments?user.id=76c9c4de896d4bc5a6e4f5bb08d20721&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:56.726472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3d18-ac95-4b14-896a-314cb14916ea None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.770598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.771503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.771783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.772055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.773341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.774831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.777373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 113/232] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:56 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:05:56.783042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:56.926585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ede91-4a74-4a38-92dd-55ee557c6157 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:56.929016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:56.975530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:56.976418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:56.976750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:56.976966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:56.978152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:56.980428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe78a7d-f121-4f27-9774-156f7943895a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:56.982425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 120/233] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:56 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:56.987733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:57.084854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:57.091171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6345060-d1fc-45b4-bcea-712ca4bd3d09 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:57.134286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:57.135213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:57.135499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:57.135774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/users/99d6a6d18f3849f1af29c47b794d01a9/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:57.142120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, user_id=99d6a6d18f3849f1af29c47b794d01a9, role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:57.143584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c5a6c-7df8-4084-91f0-f4526cec66ff None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:57.169977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 114/234] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:05:56 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/users/99d6a6d18f3849f1af29c47b794d01a9/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:05:58.098147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790545d-69fa-47d6-b980-44d97e0278a8 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.098690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790545d-69fa-47d6-b980-44d97e0278a8 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.098690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790545d-69fa-47d6-b980-44d97e0278a8 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.100051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 121/235] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:58.106104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.106490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.106670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.133359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:58.134555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:58.222596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 115/236] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:58.244719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb8bb5-e87b-4318-b8f2-74fcaf3973df None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.245011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb8bb5-e87b-4318-b8f2-74fcaf3973df None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.245291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb8bb5-e87b-4318-b8f2-74fcaf3973df None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.246917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 122/237] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:58.257540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:58.286123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.286412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.286691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.304066 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:58.305570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:58.374191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:58.404099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b0ac8-df9f-452a-9d4d-6ec30b52a5a6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:58.443987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 123/238] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:58.456077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:58.457227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.457730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.458200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.459581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:58.461513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:58.464437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 116/239] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:05:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:58.473659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:58.509565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:58.601960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d795a31-9323-4fc1-a11e-b445732b9425 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:58.604010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:58.643689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:58.644776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.645357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.648075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.649020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:58.650704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:58.688743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 124/240] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:05:58 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:05:58.697740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5db37-076e-4b75-a337-a5469938771f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:58.736526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:58.737707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.738218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.738690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.751171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:58.759637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:58.761258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:05:58.762031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:05:58.764810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 117/241] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:58.776796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:58.922960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c45dec-430f-4c31-8567-92c3c907b3ae None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:58.959591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:58.960267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:58.960579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:58.960896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:58.961655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:05:58.961998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:58.963946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:58.972730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 125/242] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:05:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:58.978231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:58.981919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.099636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fe9f9-45cc-4d27-b337-c847feab88ff None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:59.141106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:59.142047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.142367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.142589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] PATH_INFO: `/v3/users/971d25273eef4e408e60a4b18d2e0d6a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.157254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] RBAC: Authorizing `identity:get_user(user_id=971d25273eef4e408e60a4b18d2e0d6a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:59.158723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe49618-681c-4c49-bc5a-bae83ab533d4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:59.164227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 118/243] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:05:58 2022] GET /identity/v3/users/971d25273eef4e408e60a4b18d2e0d6a => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:59.166680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:59.189158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.273874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1213c5-b4a4-4050-8ec7-754823d6084b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:59.305405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2028c9c-74bb-47d7-8800-ba5f8043f9d0 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.305698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2028c9c-74bb-47d7-8800-ba5f8043f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.305981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2028c9c-74bb-47d7-8800-ba5f8043f9d0 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.307411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 119/244] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:05:59.313154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c801ba-3e8a-4386-9176-436e5b9dd43b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.313441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c801ba-3e8a-4386-9176-436e5b9dd43b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.313726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c801ba-3e8a-4386-9176-436e5b9dd43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.317705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:59.318634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.319090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.319561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.340962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a1c801ba-3e8a-4386-9176-436e5b9dd43b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:59.342156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a1c801ba-3e8a-4386-9176-436e5b9dd43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:59.405957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:59.407889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:59.416828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.434603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 120/245] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:59.435947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669860d-048e-4a5e-af80-0108a5035cb1 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.436242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669860d-048e-4a5e-af80-0108a5035cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.436579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669860d-048e-4a5e-af80-0108a5035cb1 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.437183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.438003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 121/246] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:05:59.443380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.443626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.443857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.475044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:05:59.475044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:05:59.555453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 126/247] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:05:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:05:59.556776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:59.600017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 122/248] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:05:59.601389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:59.756696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca907aad-2fcb-46a7-99af-4abbe78a2667 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:59.773771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62183c-6afa-4b8e-b729-47f61d5b6791 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:05:59.796152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.817458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:05:59.825261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:59.826418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.828263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.828775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.831410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:05:59.833892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:05:59.839716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 123/249] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:05:59 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:05:59.841778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:05:59.860195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:05:59.861551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:05:59.862035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:05:59.862521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:05:59.887458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:05:59.896758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:05:59.897631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:05:59.898400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:05:59.901477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 127/250] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:05:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:05:59.902663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.019917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4494ff-d56a-4262-8bcc-27e5f568d4a9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.043380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b1abe-4497-4b98-8dd5-33be9fc11358 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.064469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.065483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.065986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.068228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.069322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=971d25273eef4e408e60a4b18d2e0d6a) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:00.069879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:00.071571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:00.082798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 128/251] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:05:59 2022] GET /identity/v3/role_assignments?user.id=971d25273eef4e408e60a4b18d2e0d6a&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:00.083924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.122251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.123133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.123422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.123697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.151401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-173d6a9e-49e5-4667-896e-5d98d6998d2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:00.171910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:00.172918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:00.173733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 124/252] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:05:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:00.176750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c9c4-0099-4cbe-8c6e-1272f0edd1dd None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.177040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c9c4-0099-4cbe-8c6e-1272f0edd1dd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.177320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c9c4-0099-4cbe-8c6e-1272f0edd1dd None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.178798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 125/253] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:00.179579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c9c4-0099-4cbe-8c6e-1272f0edd1dd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.188584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:00.291292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:00.306742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79286fa4-a875-441f-9d66-ce3e598cb8aa None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.351623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.352398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.352688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.352968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.353731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:00.354845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:00.355870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c9c4-0099-4cbe-8c6e-1272f0edd1dd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.356255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9827eed-6ac5-4943-89cb-80313b402c91 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:00.365248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 129/254] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:00.366318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.366609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.366889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.393051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.393890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.394267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.394654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.395499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:00.395937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:00.397572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:00.405376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:00.406583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:00.407606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 126/255] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:00.408707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.543533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 130/256] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:00.545935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.578173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6b455-c3a5-4337-bdc7-c2dbabb486d3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.592020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:00.620619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.622384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.622862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.623353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.624926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:00.626931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:00.628877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 127/257] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:00.630066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.634279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:00.750617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4cdd03-192d-4f0d-bd78-bd20f5c13240 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.788836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.790180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.790655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.791146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] PATH_INFO: `/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.793167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] RBAC: Authorizing `identity:get_user(user_id=76c9c4de896d4bc5a6e4f5bb08d20721)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:00.795220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:00.797450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 131/258] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:00.798555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:00.828072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd692ffd-2059-4e8e-b001-62d5689e9636 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.905732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba17b9-b6ef-404a-97e5-d78015a6a607 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:00.908459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:00.925958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.926808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.927061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.927336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.946685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:00.947668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:00.948143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:00.948793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:00.958767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:00.967609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:00.968510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:00.969714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 128/259] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:00.970426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.027705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.029597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.093234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.154578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 132/260] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:01.155822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.164635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.249985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea894db-2337-4065-a4af-a61489a67f95 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.294800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4fe4e-6b1a-4bbc-a244-d24169edf4e6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.319661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.322088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.323348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.325136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.327466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.329638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.333669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 129/261] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:00 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:01.339379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.357497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.358521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.358992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.359478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.360529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:01.361043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.362772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.372112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 133/262] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:01.373178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.448676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.487919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d601684-e78c-4f18-a8d8-305672f9a07a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.535898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b12115-54a9-4fb6-bfb7-7be816699959 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.548696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.558882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.559817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.560216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.560520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/users/76c9c4de896d4bc5a6e4f5bb08d20721/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.563681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, user_id=76c9c4de896d4bc5a6e4f5bb08d20721, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.565219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.600238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 134/263] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:01 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/users/76c9c4de896d4bc5a6e4f5bb08d20721/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:01.601071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.604864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.605774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.606053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.606367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.608732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.615295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.617216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 130/264] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:01 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:01.625473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:01.781890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed124cc-1563-4b8a-80c0-bdbcc756f527 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.821368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.822045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.822328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.823295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.824282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=99d6a6d18f3849f1af29c47b794d01a9) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:01.824626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.826311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cc75b-ec3c-4f6f-95aa-0322a12fae99 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.833137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 131/265] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:01 2022] GET /identity/v3/role_assignments?user.id=99d6a6d18f3849f1af29c47b794d01a9&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:01.834604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2432-40e0-402e-847b-31a732510923 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.834919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2432-40e0-402e-847b-31a732510923 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.835181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c2432-40e0-402e-847b-31a732510923 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.836557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 132/266] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:01.841914 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f603-17b7-40ee-a2fd-d8a9f990a2af None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.842202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f603-17b7-40ee-a2fd-d8a9f990a2af None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.842480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f603-17b7-40ee-a2fd-d8a9f990a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.893608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7872f603-17b7-40ee-a2fd-d8a9f990a2af None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:01.894573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99640-9493-475f-90be-aa264617e68b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:01.894958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7872f603-17b7-40ee-a2fd-d8a9f990a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:01.936275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.958529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:01.974891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:01.975788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.976077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.976384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] PATH_INFO: `/v3/users/99d6a6d18f3849f1af29c47b794d01a9` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.977650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] RBAC: Authorizing `identity:get_user(user_id=99d6a6d18f3849f1af29c47b794d01a9)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:01.979189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916cdff-8cd2-4cdb-a822-8f396ca9366c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:01.981163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 135/267] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:01 2022] GET /identity/v3/users/99d6a6d18f3849f1af29c47b794d01a9 => generated 283 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:01.982415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548feb-30dc-47d2-b001-01f00883ef48 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:01.982700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548feb-30dc-47d2-b001-01f00883ef48 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:01.982977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92548feb-30dc-47d2-b001-01f00883ef48 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:01.986022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 136/268] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:01.989151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92548feb-30dc-47d2-b001-01f00883ef48 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.002542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 133/269] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:02.004461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.004981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.005524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.024755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:02.026875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:02.114904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 134/270] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:02.115873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.121741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92548feb-30dc-47d2-b001-01f00883ef48 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.158922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.159508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.159807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.160111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.224964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.234164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.235664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.248838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.253458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3aecb-cb36-42ce-a520-b548fbe8a29e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.297836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.298750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.299037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.299410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.301505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.303091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.305089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 135/271] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:02.305643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.342120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 137/272] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:02.342953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.429211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f123740a-a095-433c-8909-4a7763a79b61 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.441063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.447980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81fe060-e2fd-4456-b2b1-73634bbf9901 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.473489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.474286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.474565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.474838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.486538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:02.487397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.488069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.488385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.488660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.489372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.490825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:02.494291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:02.497043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 136/273] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:02.504468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.520718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 138/274] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:02 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:02.521857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.537018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.646567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a03d9-970d-4775-bb59-18a45a078f03 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.684364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.685271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.685732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.686172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.687096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:02.687560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.689191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.699009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 139/275] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:02.704703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.724727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.741086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53517378-34ce-43ce-bdfe-17fb36d79950 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.788501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.789383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.789669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.790230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.792152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.794466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda914f-2335-46a5-8c3a-fc6b1715e524 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.796363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 137/276] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:02.800089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799627-da4a-437c-88f0-abc4fada071e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.800677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799627-da4a-437c-88f0-abc4fada071e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.800959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4799627-da4a-437c-88f0-abc4fada071e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.802566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 138/277] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:02.806741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799627-da4a-437c-88f0-abc4fada071e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:02.834589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd66cb8-53a6-4bf9-a110-e5e421220e8c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.885574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.886454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.886734 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.887005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] PATH_INFO: `/v3/users/538b654455564938ad358d1d47df816f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.910935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] RBAC: Authorizing `identity:get_user(user_id=538b654455564938ad358d1d47df816f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.913271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab29f5f-535c-4def-8d58-8a67e9c3c4ca None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:02.917224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 140/278] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/users/538b654455564938ad358d1d47df816f => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:02.918988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.919502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.919972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.934572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:02.942832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:02.944410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:02.945495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4799627-da4a-437c-88f0-abc4fada071e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:02.990579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:02.991247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:02.991527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:02.991850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:02.993238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=99d6a6d18f3849f1af29c47b794d01a9) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:02.993238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:02.995043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:03.002553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 139/279] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:02 2022] GET /identity/v3/role_assignments?user.id=99d6a6d18f3849f1af29c47b794d01a9&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:03.003421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:03.044689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:03.052757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 141/280] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:03.106981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:03.211961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1563f-bba8-475f-84fc-4097b0d9541b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:03.294249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:03.294986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:03.295190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:03.295462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:03.296313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, user.id=538b654455564938ad358d1d47df816f, scope.system=all) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:03.296631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:03.298075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:03.311786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 140/281] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 15:06:03 2022] GET /identity/v3/role_assignments?user.id=538b654455564938ad358d1d47df816f&scope.system=all&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:03.312846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147eb8b-0ba9-481c-9504-98aca1a62201 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:03.383241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:03.384072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:03.384379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:03.384658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:03.402727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:03.410728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:03.411653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:03.412468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:03.417218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 142/282] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:03.418826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:03.466596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:03.563212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85426f3-fae8-4d22-94c2-d18525a2229a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:03.605133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:03.605811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:03.606093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:03.606369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:03.607130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:03.607427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:03.608884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:03.620869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 141/283] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:03.626395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:03.712566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:03.757122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5d96-933a-4525-94ce-a801538aea7e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:03.797966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:03.799213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:03.799689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:03.800178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] PATH_INFO: `/v3/users/971d25273eef4e408e60a4b18d2e0d6a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:03.801601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] RBAC: Authorizing `identity:get_user(user_id=971d25273eef4e408e60a4b18d2e0d6a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:03.803252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:03.805100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 143/284] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:03 2022] GET /identity/v3/users/971d25273eef4e408e60a4b18d2e0d6a => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:03.811397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:03.924640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:03.948436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ef7bc-4bb3-4fd7-9d7b-048121be5956 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:03.984165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:03.985066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:03.985349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:03.985623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:03.986809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:03.988261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148b634-decc-45f1-bbe7-7f25089a27c7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:03.990263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 142/285] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:03 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:03.995838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:04.100651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a116f10-6cc5-4e34-b698-07dcee24409e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:04.128311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:04.136183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:04.141317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:04.141601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:04.141930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/users/971d25273eef4e408e60a4b18d2e0d6a/roles/c81cae8588bc4b589a64a00b7d77bffb` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:04.145062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, user_id=971d25273eef4e408e60a4b18d2e0d6a, role_id=c81cae8588bc4b589a64a00b7d77bffb)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:04.146533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793f3d-7d73-4293-8fd2-add2f775d553 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:04.163576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 144/286] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:03 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/users/971d25273eef4e408e60a4b18d2e0d6a/roles/c81cae8588bc4b589a64a00b7d77bffb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:04.532992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd370b-277b-4218-8abe-9f161f1cc6a9 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:04.533498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd370b-277b-4218-8abe-9f161f1cc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:04.533573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd370b-277b-4218-8abe-9f161f1cc6a9 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:04.534912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 143/287] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:04.540672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:04.541056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:04.541279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:04.565251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:04.566431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:04.647578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 145/288] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:04.692396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd370b-277b-4218-8abe-9f161f1cc6a9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:04.836106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd370b-277b-4218-8abe-9f161f1cc6a9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:04.882248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:04.883052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:04.883335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:04.883613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:04.901680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:04.909497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:04.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:04.911566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 144/289] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:04.917089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.056373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d7da6-9e1d-4747-b21b-8c1b3f3a056f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:05.084698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:05.094485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:05.095101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.095388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.095668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.096440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:05.096749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:05.098188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:05.112034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 146/290] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:05.117588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.136884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:05.255098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f15ad-2359-416d-8f91-e6fda4186038 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:05.302689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:05.304041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.304562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.305037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] PATH_INFO: `/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.306615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] RBAC: Authorizing `identity:get_user(user_id=76c9c4de896d4bc5a6e4f5bb08d20721)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:05.308440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f6c0-1e8f-49bd-b2e1-bd8ed492abed None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:05.311481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 145/291] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:05 2022] GET /identity/v3/users/76c9c4de896d4bc5a6e4f5bb08d20721 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:05.316139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.454355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60546-4d8e-4e24-966f-e2b0d0bf0b2e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:05.503311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:05.504257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.504762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.505205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.527043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101d02-0911-47d3-9f05-8412a28a1a04 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.527361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101d02-0911-47d3-9f05-8412a28a1a04 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.527592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb101d02-0911-47d3-9f05-8412a28a1a04 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.529236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 146/292] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:05.535276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.536400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.536889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.557601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:05.559227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:05.598124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:05.637516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 147/293] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:05.649977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:05.651501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c69fcd-88de-4906-91e3-5fe067c81904 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:05.663783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.677045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:05.782623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 147/294] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:05.783966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938e96-55ec-486a-86f9-2bc34dff3e7b None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.784254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938e96-55ec-486a-86f9-2bc34dff3e7b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.788956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42938e96-55ec-486a-86f9-2bc34dff3e7b None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.790373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 148/295] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:05.791037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938e96-55ec-486a-86f9-2bc34dff3e7b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.824728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca107c6-f3ae-4e57-b748-563a5f1f070a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:05.862007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:05.862809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.863087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.863465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.864760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:05.866208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2052598f-9396-4b54-b2ac-26f66a7dff52 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:05.868288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 148/296] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:05.870140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.870420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.870694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.885948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:05.897036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:05.898590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:05.912494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42938e96-55ec-486a-86f9-2bc34dff3e7b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:05.958835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:05.959725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:05.960134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:05.960450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:05.961773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:05.963347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:05.965233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 149/297] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:05 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:05.970993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:05.988692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 149/298] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:05.989694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.010761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.103415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d04bd-c582-4536-8541-a23a25aef180 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.137122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c4693-c93f-4806-bfce-9698e77a41d3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.145225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.146856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.147353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.147830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] PATH_INFO: `/v3/groups` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.149241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.151683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.176332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 150/299] 10.222.0.45 () {60 vars in 1134 bytes} [Tue May 24 15:06:05 2022] POST /identity/v3/groups => generated 229 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:06.176332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.181407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.183358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.183788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.184133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.184442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.185899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=76c9c4de896d4bc5a6e4f5bb08d20721) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:06.186216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.188283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335b8ab-19b5-4631-a865-335a6bad0de9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.198512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 150/300] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:05 2022] GET /identity/v3/role_assignments?user.id=76c9c4de896d4bc5a6e4f5bb08d20721&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:06.200547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db0e6-86ff-4d2a-9e46-5a2c66732e89 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.201014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db0e6-86ff-4d2a-9e46-5a2c66732e89 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.201422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2db0e6-86ff-4d2a-9e46-5a2c66732e89 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.203003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 151/301] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:06.208956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.209245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.209497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.228532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:06.230632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:06.294438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fcb59-c4fc-431f-9881-12b8c327b2f2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.306213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 152/302] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:06.329205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.331346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.334150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.334319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.334614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.334944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.347875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:06.356538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:06.357528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:06.358283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 151/303] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:06.363588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.440024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.471146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318190f-33f5-4351-8a59-9b99cb2f3db8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.505929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb93c-98af-4880-ada0-b76d10728b0f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.546047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.546857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.547137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.547411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.561439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.561665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:06.562202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.562425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.562701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.563465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:06.563775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.565322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:06.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:06.571090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:06.571934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:06.573173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 153/304] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:06.575084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 152/305] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:06.580049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.582902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.674294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.702007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34969-ead7-4b08-88e6-b1d8b76f4178 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.727847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786db3b-469e-4f07-b635-4a8c93b3abd5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.734091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.740877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.742071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.742594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.743102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] PATH_INFO: `/v3/users/5c2340fc8f2841e89391c6e597af8058` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.758024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] RBAC: Authorizing `identity:get_user(user_id=5c2340fc8f2841e89391c6e597af8058)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.760527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.762957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 153/306] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/users/5c2340fc8f2841e89391c6e597af8058 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:06.772150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.772350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.772350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.772350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.772350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.772679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:06.772757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.775153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.786797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 154/307] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:06.791800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:06.887006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc151-7e1a-489b-ad77-189c8345ebaf None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.925608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579480df-7a82-46c7-9e19-ac2eed7e20f5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:06.931724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.932929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.933206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.933477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.934584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, user.id=5c2340fc8f2841e89391c6e597af8058, scope.system=all) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:06.934584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.936544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.948156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 154/308] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/role_assignments?user.id=5c2340fc8f2841e89391c6e597af8058&scope.system=all&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:06.960364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:06.973266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:06.974113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:06.974399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:06.974683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] PATH_INFO: `/v3/users/538b654455564938ad358d1d47df816f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:06.976210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] RBAC: Authorizing `identity:get_user(user_id=538b654455564938ad358d1d47df816f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:06.977813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74b834-ab4c-4e48-9c9c-c90f2532d3f3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:06.981258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 155/309] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:06 2022] GET /identity/v3/users/538b654455564938ad358d1d47df816f => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:06.986996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:07.050246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:07.073528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517a7f-13c1-4cfc-b54e-01476ee97fee None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.073813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517a7f-13c1-4cfc-b54e-01476ee97fee None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.074089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517a7f-13c1-4cfc-b54e-01476ee97fee None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.079586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 156/310] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:07.081499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.081860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.082210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.099359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:07.100743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:07.105311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e21486-5d0c-4f72-ba67-f71d8a7b89ab None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:07.152535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:07.153721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.154198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.154642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] PATH_INFO: `/v3/system/users/538b654455564938ad358d1d47df816f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.156404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=538b654455564938ad358d1d47df816f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:07.157979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:07.169042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 155/311] 10.222.0.45 () {58 vars in 1334 bytes} [Tue May 24 15:06:06 2022] PUT /identity/v3/system/users/538b654455564938ad358d1d47df816f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:07.187183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 157/312] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:07.211072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:07.329176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a0647-4466-418f-8e22-e3641d279b0d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:07.347832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:07.375883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:07.377130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.377647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.378135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.390357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:07.398514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:07.399379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:07.400149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:07.406739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 156/313] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:07.408788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:07.515031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d05042-bcda-4751-93cb-69072f3f4a9a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:07.538484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:07.550747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:07.551455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.551688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.551958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.552741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:07.553044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:07.554417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:07.563164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 158/314] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:07.568645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:07.672377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a225b-dcfb-4628-baf2-e0825b4320a2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:07.706776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:07.707641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.707918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.708188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] PATH_INFO: `/v3/users/971d25273eef4e408e60a4b18d2e0d6a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.709408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] RBAC: Authorizing `identity:get_user(user_id=971d25273eef4e408e60a4b18d2e0d6a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:07.710864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:07.712940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 157/315] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:07 2022] GET /identity/v3/users/971d25273eef4e408e60a4b18d2e0d6a => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:07.718836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:07.823734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f816d3c-9548-4e5c-9121-f76b5cd86e21 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:07.858192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:07.859176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:07.859656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:07.860122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:07.917255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:07.932381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:07.933407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:08.033802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 159/316] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:08.039396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:08.096899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:08.147864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268497a2-1f7c-4f88-aeee-01a2bfb2f553 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:08.187380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:08.188166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:08.188486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:08.188763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:08.189952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:08.191420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7100d-4def-4be6-abcd-8b5958d8bf39 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:08.193447 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 158/317] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:08 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:08.198880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:08.283159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:08.299036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ff33d-a065-4e1a-9cb0-c91e6ad97d33 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:08.333197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:08.333847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:08.334123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:08.334396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:08.335214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, scope.project.id=fbd29823d3324725b24edce7446a7829, user.id=971d25273eef4e408e60a4b18d2e0d6a) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:08.335521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:08.336996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d20e7-a57a-4acc-bbf4-9dd896807b58 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:08.343257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 160/318] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:08 2022] GET /identity/v3/role_assignments?user.id=971d25273eef4e408e60a4b18d2e0d6a&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:08.710918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286756a-bd51-4ac6-87a1-b05bcd854d9f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:08.711425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286756a-bd51-4ac6-87a1-b05bcd854d9f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:08.711499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286756a-bd51-4ac6-87a1-b05bcd854d9f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:08.713479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 159/319] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:08.718481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:08.718849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:08.719044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:08.736477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:08.737661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:08.807189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 161/320] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:08.830098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286756a-bd51-4ac6-87a1-b05bcd854d9f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:08.934465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286756a-bd51-4ac6-87a1-b05bcd854d9f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:08.947204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:08.970141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:08.970934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:08.971214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:08.971487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:08.972660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:08.974136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522e55f-9f4a-44cb-9db2-b5fde4c867a1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:08.975907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 160/321] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:08.982117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.071147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:09.094340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5973e760-20a3-47ad-a477-8a9b488f17f3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:09.130515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:09.131193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.131475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.131745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] PATH_INFO: `/v3/groups` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.132478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:09.133920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba66658b-7b73-455e-9c21-772bd067d57c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:09.149241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 162/322] 10.222.0.45 () {60 vars in 1134 bytes} [Tue May 24 15:06:08 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:09.254894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063febf8-f458-497a-838d-3131659ec741 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.255080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063febf8-f458-497a-838d-3131659ec741 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.255480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063febf8-f458-497a-838d-3131659ec741 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.257126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 161/323] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:09.262712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.263003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.263273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.281759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:09.282961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:09.356189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 163/324] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:09.378552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063febf8-f458-497a-838d-3131659ec741 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.496563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063febf8-f458-497a-838d-3131659ec741 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:09.531388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:09.532513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.532976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.533424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.544936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ea181626-97b3-4335-ab6d-8519f7a0af3f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:09.552866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:09.553718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:09.554531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:09.555370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 162/325] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:09.563119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.577635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686db-cc93-4a6c-bc39-351253a04d4d None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.577933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686db-cc93-4a6c-bc39-351253a04d4d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.578177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972686db-cc93-4a6c-bc39-351253a04d4d None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.579821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 163/326] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:09.585589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.585876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.586308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.605600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:09.606279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:09.647066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:09.654469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:09.692781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3831358-9da3-4c3f-a7d6-04456e7cd242 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:09.692971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 164/327] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:09.719180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.734889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:09.736530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.737486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.737993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.739200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:09.739943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:09.742012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:09.754831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 164/328] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:09.761040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.842547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5c61-fd79-4f0d-8203-7ec91e8d1c2c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:09.888538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:09.890205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.890684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.891150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.891921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa62758-ec0a-4e28-98bb-f75878df32df None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:09.903451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:09.914523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:09.915403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:09.916750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:09.920514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 165/329] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:09.926635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.929621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:09.938125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:09.939011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:09.939288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:09.939555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] PATH_INFO: `/v3/users/5c2340fc8f2841e89391c6e597af8058` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:09.940800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] RBAC: Authorizing `identity:get_user(user_id=5c2340fc8f2841e89391c6e597af8058)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:09.942252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:09.948440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 165/330] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:09 2022] GET /identity/v3/users/5c2340fc8f2841e89391c6e597af8058 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:09.949716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:09.968631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:10.046637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb1040-301e-4df3-9910-4b1c886c9b3f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:10.064264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e43f4b-d577-448e-9113-053edca91778 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:10.095950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:10.096977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:10.097481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:10.097961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:10.098723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:10.099027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:10.100469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:10.110681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 166/331] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:10.115941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:10.118204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:10.119487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:10.119913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:10.120459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] PATH_INFO: `/v3/system/users/5c2340fc8f2841e89391c6e597af8058/roles/c81cae8588bc4b589a64a00b7d77bffb` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:10.122214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5c2340fc8f2841e89391c6e597af8058, role_id=c81cae8588bc4b589a64a00b7d77bffb)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:10.123821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:10.140537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 166/332] 10.222.0.45 () {58 vars in 1334 bytes} [Tue May 24 15:06:09 2022] PUT /identity/v3/system/users/5c2340fc8f2841e89391c6e597af8058/roles/c81cae8588bc4b589a64a00b7d77bffb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:10.233504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:10.244520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56635fc2-faab-4c67-9e16-fddeefa9528b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:10.287322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:10.288205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:10.288507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:10.288775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] PATH_INFO: `/v3/users/538b654455564938ad358d1d47df816f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:10.290037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] RBAC: Authorizing `identity:get_user(user_id=538b654455564938ad358d1d47df816f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:10.292072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610ee67-ef27-437e-82c0-a93dc4b4a618 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:10.293717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 167/333] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:10 2022] GET /identity/v3/users/538b654455564938ad358d1d47df816f => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:10.299598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:10.384490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:10.432275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2d91-4c3d-4095-9f60-a0d20a585914 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:10.477440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:10.478115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:10.478395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:10.478720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:10.479802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] RBAC: Adding query filter params (role.id=82e7d694fe0846dda33cc001bd87aa4a, user.id=538b654455564938ad358d1d47df816f, scope.system=all) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:10.480121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:10.481639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3943bb-1dd5-4a07-8463-c6e54ddf9d7a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:10.494634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 167/334] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 15:06:10 2022] GET /identity/v3/role_assignments?user.id=538b654455564938ad358d1d47df816f&scope.system=all&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:12.461869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cab1-df95-4b3c-886b-74857f806a62 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.462357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cab1-df95-4b3c-886b-74857f806a62 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.462357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89cab1-df95-4b3c-886b-74857f806a62 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.464585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 168/335] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:12.469631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.470054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.470221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.493338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:12.494595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:12.528865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1444-997c-419e-bd11-fd54492d7e2b None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.529194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1444-997c-419e-bd11-fd54492d7e2b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.529421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca1444-997c-419e-bd11-fd54492d7e2b None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.530792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 169/336] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:12.536619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.537169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.537647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.564789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:12.566385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:12.576327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 168/337] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:12.612426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:12.673988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 170/338] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:12.709432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:12.716498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:12.784643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41df82c-0945-4372-a46d-c2a077711988 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:12.838632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:12.841636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:12.845408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.845690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.845968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.851183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16cee3-40a9-45f5-b586-c2c112851e48 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:12.867346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:12.876103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:12.877033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:12.882929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 169/339] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:12.897277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:12.899950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:12.901675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:12.902177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:12.902748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:12.914897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:12.923807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:12.924768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:12.925546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:12.928036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 171/340] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:12.937295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.100508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ab333-c728-4f84-9e3d-43e58576ea4d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.119787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.161135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fc866-1a63-4d0b-b031-90370f961c25 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.164377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.165057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.165333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.165602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.167000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:13.167307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:13.168735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc5f9b-802e-4af3-b6c9-e190bdc73887 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:13.185121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 170/341] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:13.186265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d90b-53f0-4f28-bac7-00d77a7a8fed None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.186551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d90b-53f0-4f28-bac7-00d77a7a8fed None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.186824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674d90b-53f0-4f28-bac7-00d77a7a8fed None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.189170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 171/342] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:13.190652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d90b-53f0-4f28-bac7-00d77a7a8fed None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.249017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.249710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.249989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.250261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.251021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:13.251325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:13.252752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b28b02-9c15-430f-88ee-7c07e388af0f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:13.271237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 172/343] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:13.272566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.272849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.273119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.309221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674d90b-53f0-4f28-bac7-00d77a7a8fed None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.309454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:13.310611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:13.323702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.350387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.351065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.351341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.351615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.401967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.429767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:13.431258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:13.457542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 173/344] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:13.460876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 172/345] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:06:13.539246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.599699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85324ced-6be8-49bc-b181-6d708f355752 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.614006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.644386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.645046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.645322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.645593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.656790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2701b6a0-d25b-43ba-a102-660fbbca2097 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.659224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.696960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.698695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.699039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.699384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.711105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:13.719091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:13.720058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:13.721012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:13.721652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 173/346] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:13.722598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.728439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:13.730375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e0bc9-80e8-474c-ba02-fcf2f2e96857 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:13.859157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 174/347] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:13.862100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2222639c-fe09-402a-9312-8bd56f9e0608 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:13.864695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19728a-2d77-461e-bd41-af25f63cef09 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.864985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19728a-2d77-461e-bd41-af25f63cef09 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.865261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb19728a-2d77-461e-bd41-af25f63cef09 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.866692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 175/348] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:13.867700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e57d-0750-4b19-a036-55dba4a333c0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.868043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e57d-0750-4b19-a036-55dba4a333c0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.868347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7471e57d-0750-4b19-a036-55dba4a333c0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.870738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 176/349] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:13.873147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e57d-0750-4b19-a036-55dba4a333c0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.906827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:13.915133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:13.916286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:13.916615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:13.916891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:13.918093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:13.922795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:13.922795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 174/350] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:13.923654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:13.930294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.009982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7471e57d-0750-4b19-a036-55dba4a333c0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.053093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935f60f-a07e-4fcb-844c-4c503100a319 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.055940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.056831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.057238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.057617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.058486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:14.058898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.060525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93623-8e63-48cd-bbca-d0f13b7e3c29 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.073328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 177/351] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.073328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.073547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.073622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.093028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:14.094219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:14.107201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.108077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.108386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.108663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.109898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.111303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d0354f-17ab-4e89-965d-59431742e067 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.113299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 175/352] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:13 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.114455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.114738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.115028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.135347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:14.136604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:14.185295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 178/353] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:14.185582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.215592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.221412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 176/354] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:14.223637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.254376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.340769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7087e-db59-4300-ac64-c6abd8cffe85 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.354410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aae41-f32c-4f00-b75e-5064fe01b7c0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.379213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.380029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.380357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.380618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.390317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.392355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.394265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 179/355] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.395209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.401777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.402528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.402879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.403220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.481696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.485910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.486824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.524554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.529469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37cf51-4a9d-4d32-8d9d-5544f3afd971 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.568123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.568772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.569055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.569330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.570503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.571958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.573896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 180/356] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.574493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.610628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 177/357] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:14.612008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.701889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cac6-1bc3-4026-a9c3-f1abedf382b3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.724938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.731939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f681d4-f3f6-4eb0-868c-f149c4f036e1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.742914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.743710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.744338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.744560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.756834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:14.764853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:14.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 181/358] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:14.767017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.776214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.777143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.777422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.777696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.788238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a743c5db-e4fb-4872-b2d8-c457f92d6183 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:14.796962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:14.797853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:14.798708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 178/359] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:14.799394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e32dd-6de1-49c0-a276-99ea3a44106b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.799682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e32dd-6de1-49c0-a276-99ea3a44106b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.799989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e32dd-6de1-49c0-a276-99ea3a44106b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.800615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:14.802552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 179/360] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:14.802925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e32dd-6de1-49c0-a276-99ea3a44106b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.888842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e87445-9e24-4f93-bc48-2d407b8928a7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.926281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e32dd-6de1-49c0-a276-99ea3a44106b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:14.930198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.931108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.931988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.931988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.932856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=81e587db604e403584fcdc9ec139396c) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:14.933166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.934794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.940503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 182/361] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.941373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:14.970866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:14.971605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:14.972179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:14.972491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:14.973529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=81e587db604e403584fcdc9ec139396c) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:14.973846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:14.976237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:14.986732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 180/362] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:14.986732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.022990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.066821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.082897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663b408-adc6-4dca-80c6-100f7ce38b56 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.135969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.136947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.137225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.137548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.138958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.140534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.142655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 183/363] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.143366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e4135-ac1b-449b-94a8-cc9805887ffd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.143483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.196673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.197957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.198237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.198694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.199491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:15.199799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.201247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc72fd-f773-49d5-8c10-f5908f90f154 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.215615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 181/364] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.218112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.218400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.218671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.239202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:15.240222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:15.285622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b9e3b-9aaa-4fd9-a8f1-d6c9ed23f8ec None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.330493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.331170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.331455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.331729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.332670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:15.332978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.334995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.342592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 182/365] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:15.343933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.349321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 184/366] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.349321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.372070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.373944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.480171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad975873-2c36-49b1-945c-457a591405b2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.483873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45565e-afee-4dda-843d-e5974ce3366f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.523751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.524202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.524669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.524960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.525130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.525252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] PATH_INFO: `/v3/users/5c2340fc8f2841e89391c6e597af8058` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.525413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.525673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.526603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] RBAC: Authorizing `identity:get_user(user_id=5c2340fc8f2841e89391c6e597af8058)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.526956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.528081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.528410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.530047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 183/368] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.530272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 185/368] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/users/5c2340fc8f2841e89391c6e597af8058 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.530769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.530979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.622312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.640637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.641014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b9c5c-8691-42c9-9d98-902ff51dd2f0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.657488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3997ed-4a3f-4645-838e-617bd64fc26e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.677779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.678437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.678718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.678993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.697554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.698855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.699339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.699821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.711441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:15.721222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:15.722085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:15.722866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:15.730878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 186/369] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:15.731960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.751524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.753010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.847230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:15.859983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3ffa6-0c77-4c90-8159-2b61cfbf9205 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:15.880145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 184/370] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:15.881611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.908618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:15.909316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:15.909597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:15.909871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:15.911236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=fbd29823d3324725b24edce7446a7829) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:15.911557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:15.913307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:15.924470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 187/371] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:15.925356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:15.965912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:16.014621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9121b-d426-47a7-98ca-4a58b5c65493 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.062993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.063675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.063957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.064232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.065116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] RBAC: Adding query filter params (role.id=c81cae8588bc4b589a64a00b7d77bffb, user.id=5c2340fc8f2841e89391c6e597af8058, scope.system=all) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:16.065428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.066921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.089536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 185/372] 10.222.0.45 () {56 vars in 1308 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/role_assignments?user.id=5c2340fc8f2841e89391c6e597af8058&scope.system=all&role.id=c81cae8588bc4b589a64a00b7d77bffb => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:16.101552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:16.184874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee140f9-b4b2-4468-9e1e-436fc3b0621d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.205321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:16.266187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.267247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.267753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.268251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.269259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:16.269763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.271375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.288444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 188/373] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:16.290244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:16.349019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc88a-44fc-4c4d-8bc9-c800078881c3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.434852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:16.445396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.446527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.446981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.447444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.453116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.454858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.456965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 186/374] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:16 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:16.463112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:16.487730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae14f0f-9830-432b-be26-ede95b71a247 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.556104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.557570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.558003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.558447 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.651971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c57f93-d41d-4963-8a4e-e9b2681d88a8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.669583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:16.701240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.702838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.710244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.711297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.711728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.712145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.719170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.720865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.727123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 187/375] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:16 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:16.731614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:16.868685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:16.914623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 189/376] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:06:16.917201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:16.934994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050144f0-16f7-47a7-b9cd-6424c7d3d186 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:16.977620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:16.979131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:16.979822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:16.980322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:16.981453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=fbd29823d3324725b24edce7446a7829) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:16.981916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:16.983518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:16.989592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 188/377] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:16 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:17.120752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf8d50-a8d6-48c2-a628-b47acc320204 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:17.161914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:17.162772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.163049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.163321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] PATH_INFO: `/v3/projects/afcdba301526446ba48fe1e8c0bb21bc` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.169200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] RBAC: Authorizing `identity:get_project(project_id=afcdba301526446ba48fe1e8c0bb21bc)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:17.170710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:17.172690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 190/378] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:16 2022] GET /identity/v3/projects/afcdba301526446ba48fe1e8c0bb21bc => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:17.178300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:17.257713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:17.321528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a828217-5239-45c6-9627-aece4afbf0a4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:17.366070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:17.366949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.367228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.367498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] PATH_INFO: `/v3/groups/66aef2f4a06e402982523d56e7a5bffe` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.379247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] RBAC: Authorizing `identity:get_group(group_id=66aef2f4a06e402982523d56e7a5bffe)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:17.380788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d071463-b25e-4ddb-a5c3-d42440ed4634 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:17.382928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 189/379] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:17 2022] GET /identity/v3/groups/66aef2f4a06e402982523d56e7a5bffe => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:17.389240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:17.449088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:17.519041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4519a-7102-44c4-9002-e2dc696d9f75 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:17.569123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:17.570210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.570714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.571188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.572844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] RBAC: Adding query filter params (group.id=66aef2f4a06e402982523d56e7a5bffe, role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=afcdba301526446ba48fe1e8c0bb21bc) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:17.573382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:17.575041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e966e-0784-4456-94c4-e1d09875e585 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:17.581789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 191/380] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:17 2022] GET /identity/v3/role_assignments?group.id=66aef2f4a06e402982523d56e7a5bffe&scope.project.id=afcdba301526446ba48fe1e8c0bb21bc&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:17.623818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62647-6acb-4733-944d-a7591bb9de27 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.624112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62647-6acb-4733-944d-a7591bb9de27 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.624402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d62647-6acb-4733-944d-a7591bb9de27 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.626048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 190/381] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:17.631647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e864-b820-4cee-bd23-93d47a67517b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.632155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e864-b820-4cee-bd23-93d47a67517b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.632651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5e864-b820-4cee-bd23-93d47a67517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.657062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f9f5e864-b820-4cee-bd23-93d47a67517b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:17.658550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f9f5e864-b820-4cee-bd23-93d47a67517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:17.739140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 192/382] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:17.773714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d62647-6acb-4733-944d-a7591bb9de27 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:17.781993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063887-1323-49bc-800d-4fdea5a21509 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.782300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063887-1323-49bc-800d-4fdea5a21509 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.782557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e063887-1323-49bc-800d-4fdea5a21509 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.784070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 193/383] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:17.790077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.790538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.791463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:17.809527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:17.810814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:17.862261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:17.909700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 194/384] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:17.933051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:17.936993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d62647-6acb-4733-944d-a7591bb9de27 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:17.939439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:17.995848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:17.997100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:17.997615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:17.998091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.015008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:18.023289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:18.024220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:18.025072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 191/385] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:18.032814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.111046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7ae6b-f9cf-4ec1-8cc2-316b3b141290 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.174364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.175168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.175443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.175707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.199596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:18.207597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:18.208508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:18.209698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 195/386] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:18.214661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.231977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ae2289-3879-4a10-ab69-0d14d15fb6dd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.258639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:18.308705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.309295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.309573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.309843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.310591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:18.310895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.312280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.321085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 192/387] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:18.327283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.360392 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:18.399242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ec260-aa59-403c-bf0c-630de32f1b94 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.465417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.468533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.469166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.470375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.471345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:18.472041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.474662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.487743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 196/388] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:18.493289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.499569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e008a2-dce0-40d8-b89b-9aa5441dd8f7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.544575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.546206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.546744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.547164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.550157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.551945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.556246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 193/389] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:18.561273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.610688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:18.640961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed1abe-39bf-4b78-be60-3aea704216eb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.709444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.710316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.710597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.710870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.718441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.719958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.721631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 197/390] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:18.726816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.776640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:18.790430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87142d3-9bb7-4c48-a3e2-991cae412dca None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.844418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5263e23-505b-4091-baa7-da31c1329c9a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:18.851529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.852588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.853043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.853497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.855094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.856726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07852-193b-45a4-8030-e54de525ed86 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.858570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 194/391] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:18.860411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ff805-4635-4b0d-b27c-a5511ca99b9e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.860900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ff805-4635-4b0d-b27c-a5511ca99b9e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.861322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ff805-4635-4b0d-b27c-a5511ca99b9e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.862695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 195/392] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:18.870272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.870875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.871309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.895054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:18.895974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:18.899444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:18.899444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:18.901021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:18.902681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:18.904023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:18.904810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 198/393] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:18 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:18.905976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:18.911953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:18.937060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:19.000337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 196/394] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:19.000337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.042623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea375-a8c9-4239-860c-e372b31ced17 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.091578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:19.092844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.093269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.093712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.096261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, group_id=13cae53fe4b64443a8f7d32331e1dd6f, role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:19.098119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:19.109241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:19.124325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 199/395] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 15:06:18 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:19.124325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.128258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbfa44-6486-4fcc-8b25-c7d2bf40593f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.204479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:19.206846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.207128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.207401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.214000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] RBAC: Authorizing `identity:create_grant(project_id=81e587db604e403584fcdc9ec139396c, group_id=13cae53fe4b64443a8f7d32331e1dd6f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:19.218861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:19.247843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 197/396] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 15:06:18 2022] PUT /identity/v3/projects/81e587db604e403584fcdc9ec139396c/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:19.285217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:19.297908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e0cec-bc11-4e98-9078-63616fb35afe None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.338806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:19.339635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.339915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.340188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.352113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:19.360666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:19.361555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:19.362422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 200/397] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:19.369867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.487955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d95b7d-e48d-498d-80ab-5ac3dda7ad14 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.538793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:19.539471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.539749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.540019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.541331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] RBAC: Adding query filter params (name=member) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:19.541638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:19.543007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56bbef2-11e5-4bcd-9037-f5fba0b6f655 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:19.560430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 198/398] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:19.566484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.687572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048fc1-fd2e-4805-bb70-d33e4d5e210b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.687756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11d6db5-31e1-49b1-a432-fa17b467b3fc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.687846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048fc1-fd2e-4805-bb70-d33e4d5e210b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.688066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a048fc1-fd2e-4805-bb70-d33e4d5e210b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.689821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 199/399] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:19.697325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.697744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.698119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.716154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:19.717498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:19.730333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:19.731346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:19.734034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:19.734322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:19.734602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:19.736473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:19.737963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:19.739879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 201/400] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:19 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:19.746358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.770839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:19.803918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 200/401] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:19.827989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:19.954266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90efbf52-11ce-47eb-9fb7-feabf6f9722c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:19.971908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4304205-26d5-4461-aa57-ba28a6865533 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.014385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.015307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.015594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.015872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.024196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.025826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.026292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.026764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.027062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:20.033031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.034691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:20.035248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:20.036100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:20.036911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:20.039457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 202/402] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:19 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:20.040822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 201/403] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:20.046110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.046561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.053446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.190244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.205117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037ec14-0874-4358-bb5d-00898267c347 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.212470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f1960-ea55-4339-adad-aeed5532f7c1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.244457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.245405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.245843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.246256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.247159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:20.247612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.248726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.249390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b18dc1-2361-436f-ae81-fa5fe8c6bb71 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.249794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.250180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.250549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.252647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, group_id=13cae53fe4b64443a8f7d32331e1dd6f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.254168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51048a91-2731-4b05-9c33-1d6e03414db5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.261276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 202/404] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:20.262817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7ebd-1042-4496-8cfc-8033cfb28c7a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.263280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7ebd-1042-4496-8cfc-8033cfb28c7a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.263751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc7ebd-1042-4496-8cfc-8033cfb28c7a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.265179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 203/405] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:20.270303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7ebd-1042-4496-8cfc-8033cfb28c7a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.277038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 203/406] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 15:06:20 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:20.284347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.284347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.284347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.307438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:20.309044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:20.393868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 204/407] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:20.428705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.433960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.499865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc7ebd-1042-4496-8cfc-8033cfb28c7a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.538406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.539801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.540200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.541458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.541937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.543366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.545125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.547131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 204/408] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:20.552840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.575970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382dddc0-9c20-4c6b-8cc3-72e1f1d8e743 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.615001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.615996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.616376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.616803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.628643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:20.637711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:20.638669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:20.639809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 205/409] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:20.645688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.660819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b118f-cc4a-4d27-b1a0-d24a3a191ce2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.695178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.696039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.696340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.696615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.698356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.699800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.701831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 205/410] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:20.707203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.710063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.757831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c8f5c-0b03-4bf7-ada3-693e82ac958f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.792278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:20.801378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.803887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.804383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.804986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.805968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:20.806808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.809029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.827370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 206/411] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:20.835251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:20.892444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d21d3-9c1b-47ab-89c0-59b5b68fbb88 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:20.938755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:20.940005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:20.940117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:20.940343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:20.942599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] RBAC: Authorizing `identity:create_grant(project_id=fbd29823d3324725b24edce7446a7829, group_id=13cae53fe4b64443a8f7d32331e1dd6f, role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:20.944239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:20.971990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 206/412] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 15:06:20 2022] PUT /identity/v3/projects/fbd29823d3324725b24edce7446a7829/groups/13cae53fe4b64443a8f7d32331e1dd6f/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:21.003202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0a966-0a73-44c0-a4df-95a6106ab89c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:21.041517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:21.042321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:21.042605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:21.042884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] PATH_INFO: `/v3/groups/66aef2f4a06e402982523d56e7a5bffe` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:21.044171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] RBAC: Authorizing `identity:get_group(group_id=66aef2f4a06e402982523d56e7a5bffe)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:21.045767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:21.047849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 207/413] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:20 2022] GET /identity/v3/groups/66aef2f4a06e402982523d56e7a5bffe => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:21.053425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:21.170934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:21.223493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f37a3-6402-4008-a35c-14186d025dfb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:21.257871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:21.258733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:21.259004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:21.259275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] PATH_INFO: `/v3/projects/afcdba301526446ba48fe1e8c0bb21bc` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:21.260535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] RBAC: Authorizing `identity:get_project(project_id=afcdba301526446ba48fe1e8c0bb21bc)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:21.261983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af283fb-ae8b-44cd-b205-902439dd961d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:21.263958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 207/414] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:21 2022] GET /identity/v3/projects/afcdba301526446ba48fe1e8c0bb21bc => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:21.269360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:21.352849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:21.406458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8e287-b4d3-4cb8-b2be-e32d687c024f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:21.450095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:21.450988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:21.451267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:21.451542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] PATH_INFO: `/v3/projects/afcdba301526446ba48fe1e8c0bb21bc/groups/66aef2f4a06e402982523d56e7a5bffe/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:21.454179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] RBAC: Authorizing `identity:create_grant(project_id=afcdba301526446ba48fe1e8c0bb21bc, group_id=66aef2f4a06e402982523d56e7a5bffe, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:21.456434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc77c4-0cf2-45e9-95e3-862435372229 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:21.475848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 208/415] 10.222.0.45 () {58 vars in 1442 bytes} [Tue May 24 15:06:21 2022] PUT /identity/v3/projects/afcdba301526446ba48fe1e8c0bb21bc/groups/66aef2f4a06e402982523d56e7a5bffe/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:22.127926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2786-753a-4837-b7c3-8886fdf49700 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.128917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2786-753a-4837-b7c3-8886fdf49700 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.129420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a2786-753a-4837-b7c3-8886fdf49700 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.131248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 208/416] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:22.137373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4834-e28b-436d-ac2a-b9c698497f43 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.138186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4834-e28b-436d-ac2a-b9c698497f43 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.138664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a4834-e28b-436d-ac2a-b9c698497f43 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.140475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 209/417] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:22.141976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.142469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.142934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.145485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.145966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.146448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.172864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:22.173639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:22.174607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:22.174806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:22.268474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 209/418] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:22.290736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 210/419] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:22.305520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.331603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.415559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:22.429879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:22.495613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa713089-b0ba-4ce6-af00-2f5c6138dda0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:22.540185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aa9f98-b633-4146-a9a3-a9c1601cb11a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:22.561789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:22.562800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.563084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.563361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.589446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:22.603399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:22.604369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:22.605420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:22.606487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 210/420] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:22.615031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.622155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:22.622961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.623243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.623517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.647280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:22.663196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:22.664170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:22.664985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:22.672443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 211/421] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:22.688394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.802987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f22650-fad0-4a39-80c6-9e5b317eb826 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:22.807749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ea324-06c5-45a9-9a1c-0916209e006a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:22.819116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:22.827742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:22.842907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:22.843630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.844009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.844411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.845466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:22.845869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:22.847562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef4f9b-5c21-4199-aa9a-5315315b6956 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:22.848178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:22.849135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.849587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.850097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.850994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:22.851442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:22.852984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:22.857447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 211/422] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:22.862699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 212/423] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:22.863690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.863829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7ee7d-e8a8-4e0b-b270-2c8dcaf60f85 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:22.864155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7ee7d-e8a8-4e0b-b270-2c8dcaf60f85 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:22.864544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7ee7d-e8a8-4e0b-b270-2c8dcaf60f85 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:22.870697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 212/424] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:22.870697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7ee7d-e8a8-4e0b-b270-2c8dcaf60f85 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:22.974856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea7635-e1d3-4801-b2ec-ca7133318daf None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:22.976997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7ee7d-e8a8-4e0b-b270-2c8dcaf60f85 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.012490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.013382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.013821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.014100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.014904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.015761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.016260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.016857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.058045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.086516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.093252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.094607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69e7146-093a-4bf9-a2ce-1cd5869de364 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.116457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.118673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.197885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 213/425] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:23.199293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.199579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.199846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.215729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:23.216968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:23.249646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 213/426] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:23.250503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.316707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 214/427] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:23.316707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.336652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.354939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.375178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f61b-6de5-4ed6-b9e2-258c9b6d4ce2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.410200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.411058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.411332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.411599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.412808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.414427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.416950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 214/428] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.418035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.456020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939c5cf-c918-4d7e-aee8-13136955f99a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.497330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.498193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.498474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.498748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] PATH_INFO: `/v3/projects/81e587db604e403584fcdc9ec139396c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.499937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] RBAC: Authorizing `identity:get_project(project_id=81e587db604e403584fcdc9ec139396c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.501440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.504448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 215/429] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/projects/81e587db604e403584fcdc9ec139396c => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.504448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.524110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b6ae8-686b-4e0b-8676-44fac05c714f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.560896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.561688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.561972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.562239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.573133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:23.573847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a5785471-28f9-4b61-92da-3b640906bdd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:23.582300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:23.583393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:23.584205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 215/430] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:23.584893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294700-cf8e-41e0-a939-6032644c99c8 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.584990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294700-cf8e-41e0-a939-6032644c99c8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.585265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b294700-cf8e-41e0-a939-6032644c99c8 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.586653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 216/431] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:23.587167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b294700-cf8e-41e0-a939-6032644c99c8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.601519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.616451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f2e13-83a4-4616-b87f-18cc907f319d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.658638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.659688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.660825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.661108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.662282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.663736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.671225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 216/432] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.671225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.695867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b294700-cf8e-41e0-a939-6032644c99c8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.732793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.733652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.733933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.734203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.735367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.736852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f1ab88-b912-4905-80c0-4088644534d0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.738608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 217/433] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.744477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.744784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.745075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.761850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:23.763037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:23.779541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53595181-db2b-4e0b-92be-eb74a6f6f386 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.817034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.817861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.818184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.818749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.819503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] RBAC: Adding query filter params (name=member) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:23.819809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.821756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.828506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.837491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 217/434] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.838360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.843836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 218/435] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:23.844801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:23.863524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:23.944021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f5fc8-3d91-4813-86b4-a20eebe79b05 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.959218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ad724-c975-476d-b4bf-1838949f2949 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:23.977508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:23.978667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:23.978948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:23.979217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:23.980331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=81e587db604e403584fcdc9ec139396c) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:23.980639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:23.982093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:23.992347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 218/436] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:23.992347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.007741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.008520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.008801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.009247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.010509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=81e587db604e403584fcdc9ec139396c) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:24.010823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.012353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.024526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 219/437] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=81e587db604e403584fcdc9ec139396c&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:24.025606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.101597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.161362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59494-b9f1-4871-ba19-b98edff86e2d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.189470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.201226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.202549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.203010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.203464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.247317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408b573-cfad-4dd6-9dcb-14d630c88408 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.298969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.299877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.300067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.300372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.307309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.308776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c874f-2115-42b6-b4ac-0a900367d38e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.312809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-12debe6f-e71e-47fd-888d-8a5edb209548 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:24.326454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:24.327363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:24.328134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 220/438] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:24.329478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186bec-aa0c-464b-a1c3-f2431515b338 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.329782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186bec-aa0c-464b-a1c3-f2431515b338 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.330003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5186bec-aa0c-464b-a1c3-f2431515b338 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.332374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 221/439] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:24.333325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5186bec-aa0c-464b-a1c3-f2431515b338 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.385786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.433726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 219/440] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:24.435451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.435451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.435692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.454233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:24.456552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:24.457837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5186bec-aa0c-464b-a1c3-f2431515b338 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.492209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.498977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.499846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.500131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.500441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.501731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:24.502042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.503442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.515727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 222/441] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:24.517561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.542657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 220/442] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:24.542657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.639994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bda1-ab34-4d99-8766-a123a7820bc7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.649676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.659825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa40ac1-214c-493c-92b4-38cfbf822589 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.678785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.679651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.679929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.680205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.681460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.682902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.684623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 223/443] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:24.685387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.703186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.703840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.704122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.704475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.746968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.772657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.774139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.785322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebdc00-7cc0-4fbf-9f11-a26a2e25703c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.817488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.818269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.818548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.818827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.829440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:24.837021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:24.838491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:24.839259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 224/444] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:24.839899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.870116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.877867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 221/445] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:24.878508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:24.939102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda8ecf-2453-4b34-97da-4cd500fbaef4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.960797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:24.973222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:24.974063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:24.974347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:24.974623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:24.975783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:24.977054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ee931-d556-4e58-b68b-63ed323349d2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:24.977236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:24.979022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 225/446] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:24.979851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.011936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.012622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.012901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.013172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.013905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:25.014208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.015593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.023522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 222/447] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:25.024115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.078598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.084120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fcf39-436e-4a4c-a1cc-1dda7ce274d1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.117332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.118195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.118485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.119257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] PATH_INFO: `/v3/projects/fbd29823d3324725b24edce7446a7829` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.120465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] RBAC: Authorizing `identity:get_project(project_id=fbd29823d3324725b24edce7446a7829)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.121632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d7d99-50c6-4423-bd0c-1a34f22837db None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.121902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.123571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 226/448] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:24 2022] GET /identity/v3/projects/fbd29823d3324725b24edce7446a7829 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:25.124463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.155155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.155801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.156084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.156454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.157283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=82e7d694fe0846dda33cc001bd87aa4a, scope.project.id=fbd29823d3324725b24edce7446a7829) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:25.157595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.159025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.164805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 223/449] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=82e7d694fe0846dda33cc001bd87aa4a => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:25.167532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.186746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.232040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0e97d-25aa-47c1-a7cc-65e194f8e826 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.269499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.270163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.270447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.270723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.291805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a21483-d74e-4c06-9f81-f4260e4f4564 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.315548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.333865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.334736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.335019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.335296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] PATH_INFO: `/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.336755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] RBAC: Authorizing `identity:get_group(group_id=13cae53fe4b64443a8f7d32331e1dd6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.338268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.340542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 224/450] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/groups/13cae53fe4b64443a8f7d32331e1dd6f => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:25.343024 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.344518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.345894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.413519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.468613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 227/451] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:25.474853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.481435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ab8ef-8103-4531-afdd-08fc3d2a76a5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.520749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.521328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.521608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.521880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.522714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] RBAC: Adding query filter params (group.id=13cae53fe4b64443a8f7d32331e1dd6f, role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=fbd29823d3324725b24edce7446a7829) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:25.523020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.524488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.532245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 225/452] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/role_assignments?group.id=13cae53fe4b64443a8f7d32331e1dd6f&scope.project.id=fbd29823d3324725b24edce7446a7829&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:25.586585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e26a8c-4c7b-470f-8bfc-f84d5817b450 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.623140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.624009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.624287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.624589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] PATH_INFO: `/v3/projects/afcdba301526446ba48fe1e8c0bb21bc` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.625821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] RBAC: Authorizing `identity:get_project(project_id=afcdba301526446ba48fe1e8c0bb21bc)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.627294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.629043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 228/453] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/projects/afcdba301526446ba48fe1e8c0bb21bc => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:25.635374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.690612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.739675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9024423-e601-487a-9303-c6c3d4a4e066 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.775619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.776449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.776732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.777006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] PATH_INFO: `/v3/groups/66aef2f4a06e402982523d56e7a5bffe` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.778178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] RBAC: Authorizing `identity:get_group(group_id=66aef2f4a06e402982523d56e7a5bffe)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.779756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a3a44-aa54-44eb-80a0-31d20de26daa None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.781292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 226/454] 10.222.0.45 () {56 vars in 1180 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/groups/66aef2f4a06e402982523d56e7a5bffe => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:25.787249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:25.819714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:25.894482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920c246-a72c-4f27-9115-0ade6367f029 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:25.926758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:25.927404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:25.927679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:25.927946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:25.928797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] RBAC: Adding query filter params (group.id=66aef2f4a06e402982523d56e7a5bffe, role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=afcdba301526446ba48fe1e8c0bb21bc) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:25.929103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:25.930529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031aa4ae-28ba-4cc1-adfd-78f4aee4d8b7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:25.936668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 229/455] 10.222.0.45 () {56 vars in 1376 bytes} [Tue May 24 15:06:25 2022] GET /identity/v3/role_assignments?group.id=66aef2f4a06e402982523d56e7a5bffe&scope.project.id=afcdba301526446ba48fe1e8c0bb21bc&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:28.737774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bfae-bf10-4f04-bf1c-fc833d0fa64f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:28.738288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bfae-bf10-4f04-bf1c-fc833d0fa64f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:28.738394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824bfae-bf10-4f04-bf1c-fc833d0fa64f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:28.739950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 227/456] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:28.745651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:28.746139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:28.746210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:28.772792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:28.773246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022388-721c-40c6-91a2-369a25ea9748 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:28.773535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022388-721c-40c6-91a2-369a25ea9748 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:28.773811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e022388-721c-40c6-91a2-369a25ea9748 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:28.774192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:28.775701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 228/457] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:28.804781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9eaf-83eb-4455-87ae-40041fa92004 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:28.805189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9eaf-83eb-4455-87ae-40041fa92004 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:28.805472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e9eaf-83eb-4455-87ae-40041fa92004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:28.837111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5f2e9eaf-83eb-4455-87ae-40041fa92004 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:28.838831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5f2e9eaf-83eb-4455-87ae-40041fa92004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:28.885838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 230/458] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:28.922416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:28.950756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 229/459] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:28.968654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:28.987932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f10c-d34f-4193-8c69-867aa657bb75 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:28.988210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f10c-d34f-4193-8c69-867aa657bb75 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:28.988432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75f10c-d34f-4193-8c69-867aa657bb75 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:28.989795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 230/460] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:29.001247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.001535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.001818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.018535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:29.019712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:29.054142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2f28f-c0d0-40ab-9f36-ce07049e6936 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.070284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.093122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.093939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.094215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.094483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.095822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 231/461] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:29.097135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.127677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.129314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:29.155377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:29.156323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:29.157169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:29.157169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:29.157169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.157169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:29.157169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 231/462] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:29.157973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.213288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8aaae7-5b07-4bdb-8cda-c5cc1f763d32 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.233943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.252756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.253878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.254407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.254877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.266120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.266287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995406f5-d098-42f2-b06a-07bf7d4e46b0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.268115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:29.287102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:29.287945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:29.289126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:29.289126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:29.289126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.289126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:29.291427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 232/463] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:29.297193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.300598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.301321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.301602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.301873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.313919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.315697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:29.332796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:29.333666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:29.334486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:29.334486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:29.334486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:29.334486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:29.334486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 232/464] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:29.335170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.338936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.408048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c17ba-6557-41a6-8130-dec783efaf6d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.442438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fb43d-5c65-4607-8197-7dbff8094022 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.443482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.444141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.444465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.444745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.445457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:29.445763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.447141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.463255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 233/465] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:29.463255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.472242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.485489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.486129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.486406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.486687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.487555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:29.487727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.489173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a447ca5-5e60-445d-bd27-6a2ecfc9797d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.506746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 233/466] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:29.506746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d221-5806-459f-8c84-d173328e8dff None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.506746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d221-5806-459f-8c84-d173328e8dff None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.506746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d221-5806-459f-8c84-d173328e8dff None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.508427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 234/467] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:29.509522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d221-5806-459f-8c84-d173328e8dff None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.555855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.566342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f356093-7f19-4596-aef0-9efd66f86011 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.605073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.605735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.606016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.606287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.607004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:29.607396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.608715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.615998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d221-5806-459f-8c84-d173328e8dff None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.624971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 234/468] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:29.624971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.654365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.655040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.655324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.655597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.656490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.657743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6688a8c3-42a7-41d5-a789-1b026a005a52 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.679249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 235/469] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:29 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:29.680550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.680837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.681190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.702867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:29.704581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:29.745696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.750792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6eda3c-7776-47f0-8945-70a57a427e38 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.791865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 236/470] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:29.793080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.796953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.798435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.798719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.798992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.799698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:29.801262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:29.831217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 235/471] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:29 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:29.836337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:29.856906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:29.941653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc2d68-0644-48da-b81c-993083939298 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:29.996358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:29.997349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:29.997813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:29.998296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:29.999192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:30.000912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:30.016683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3722-d70b-4455-bd0d-3d98e75de55d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:30.033767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 237/472] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:29 2022] POST /identity/v3/users => generated 248 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:30.071644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:30.072579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.072915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.073194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.092955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:30.094577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ee01c227-a862-41eb-8832-e915bb9f21cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:30.115261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:30.116144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:30.117002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:30.117002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:30.117002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:30.117002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.117002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 236/473] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:30.123414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:30.155244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:30.245009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5f03c-0f00-44c7-a24b-83f3392d843f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:30.285171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:30.285844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.286180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.286402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.287144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:30.287450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:30.288885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:30.292606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da367de-65ff-49dc-8f85-e126b70d984a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.292887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da367de-65ff-49dc-8f85-e126b70d984a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.293163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da367de-65ff-49dc-8f85-e126b70d984a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.294775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 237/474] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:30.300408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.300724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.300983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.303285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 238/475] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:30.311607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:30.326333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:30.327747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:30.401443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:30.420450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 238/476] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:30.430318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:30.443927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:30.450030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc314bf-8a3c-4357-8364-9b36d0aea470 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:30.493742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:30.494567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.494848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.495218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.496060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:30.497655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:30.520027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 239/477] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:30 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:30.570570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c46f5-ad56-43c5-8e9e-be5a89e02860 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:30.616130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:30.617160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.617583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.617995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.618949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:30.620582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:30.636354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.sql.core [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:06:30.636354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 15:06:30.636354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [parameters: {'id': 'b9d924d51396432c8956b0c52c46c43a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 15:06:30.636354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80365) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application conn.query(q) May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result.read() May 24 15:06:30.658619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return ref.to_dict() May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:06:30.659854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application next(self.gen) May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application yield resource May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application next(self.gen) May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise self.value May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application session.commit() May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:06:30.661028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.session.flush() May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application compat.raise_( May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:06:30.662058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:06:30.662940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application conn.query(q) May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result.read() May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:06:30.663826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [parameters: {'id': 'b9d924d51396432c8956b0c52c46c43a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:30.664649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 15:06:30.665483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:30.668216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 239/478] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 15:06:30 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:30.674453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:30.816501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36620e32-b516-42ca-b24e-035baa781fb7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:30.853110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:30.853926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:30.854211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:30.854746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:30.865798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1078ea3e-79a0-4a6c-9904-a2652c64d67b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:30.874069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:30.876079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:30.877989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 240/479] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:06:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:30.883826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:30.942784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:30.994668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05047c4-2d9a-463f-9c7b-c809a672c17c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:31.033129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:31.033809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:31.034093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:31.034371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:31.035127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:31.035493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:31.037002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a57b3-fa4e-4fb7-b307-db95eb04f319 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:31.046152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 240/480] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:32.456046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099701c3-0840-403f-9918-bfc760165469 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.456554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099701c3-0840-403f-9918-bfc760165469 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.456554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099701c3-0840-403f-9918-bfc760165469 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.458242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 241/481] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:32.463460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf0a4-8b37-4013-a1f9-3157f303be0f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.463888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf0a4-8b37-4013-a1f9-3157f303be0f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.464063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edf0a4-8b37-4013-a1f9-3157f303be0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.489290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-88edf0a4-8b37-4013-a1f9-3157f303be0f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:32.490471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-88edf0a4-8b37-4013-a1f9-3157f303be0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:32.577225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 241/482] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:32.612955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099701c3-0840-403f-9918-bfc760165469 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:32.616774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:32.631565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89ea0-fcee-4e3e-83ee-e622c38dc7d7 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.631879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89ea0-fcee-4e3e-83ee-e622c38dc7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.632117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb89ea0-fcee-4e3e-83ee-e622c38dc7d7 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.633634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 242/483] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:32.640459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.640459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.640459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.673518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:32.674692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:32.780701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:32.833166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099701c3-0840-403f-9918-bfc760165469 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:32.834558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 243/484] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:32.870208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:32.871013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.871301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.871584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.873503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:32.907448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1b6d594a-e8c6-479a-a0b9-64d5377c6136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:32.915196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:32.916660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:32.917808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 242/485] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:32.919285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951598-5a0a-431e-9d99-0ad16ffa7d9e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:32.919285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951598-5a0a-431e-9d99-0ad16ffa7d9e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:32.919285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951598-5a0a-431e-9d99-0ad16ffa7d9e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:32.924402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 243/486] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:32.925257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951598-5a0a-431e-9d99-0ad16ffa7d9e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:32.988904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:32.993429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a49b6-035d-4f9d-b7b3-ab959ee6334f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.034215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e951598-5a0a-431e-9d99-0ad16ffa7d9e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.035832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.036733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.037087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.038168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.049246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a5cb7b70-44a4-418b-87e8-bb3e60fe6c77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:33.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:33.058704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:33.059637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:33.061729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 244/487] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:33.063097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.063481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.063934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.079578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.080259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.080594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.080871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.081628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:33.081972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.083069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:33.083320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.084679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:33.094163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 244/488] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:33.095525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.117110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.157820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 245/489] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:33.165910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.230110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2c5e0-6b8d-41e3-8463-1401643c002a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.261634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.279174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.279966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.280341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.280695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.282342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:33.282735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.284623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.297487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d9b46-6b31-4e49-86ab-efba665bef84 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.298398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 245/490] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:33.299482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.338843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.339804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.339952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.340230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.351032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.352626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:33.371615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:33.372656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:33.373584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:33.373584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:33.373584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:33.373584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:33.377378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 246/491] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:33.378221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.397788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.415237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428da167-1faa-4127-8ba4-aff6bc7006d1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.453140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.454218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.454639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.455051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.466475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:33.474837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:33.475737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:33.476590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:33.480442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 246/492] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:33.481887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.487147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df78d87-9e40-4e57-a088-6ba2a472698c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.523191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.525298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.525577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.525846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.535890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.537567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.539143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-191a65d0-0846-4cdd-88e9-8a93447e181f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:33.557360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:33.558383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:33.559363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:33.559363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:33.559363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:33.559363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:33.563166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 247/493] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:33.564512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac70a1-6b4d-4985-bceb-feb8474bdc6a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.564796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac70a1-6b4d-4985-bceb-feb8474bdc6a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.565072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac70a1-6b4d-4985-bceb-feb8474bdc6a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.568347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 248/494] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:33.568347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac70a1-6b4d-4985-bceb-feb8474bdc6a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.597083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06e0ae-c613-4c54-9e4a-3e4ed05a959d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.633311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.633969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.634763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.635037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.635749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:33.636059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.637476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.646565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.651012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 247/495] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:33.651834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.683696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac70a1-6b4d-4985-bceb-feb8474bdc6a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.720493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.721156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.721435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.721705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.722608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:33.722910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.725089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b675d-d52e-4cdb-a6b6-9038f09b8731 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.739139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 249/496] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:33.739139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.739139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.739139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.757172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:33.758439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:33.760074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92231e-985e-4520-b1a9-1010fce41685 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.770896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.801313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.801894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.802172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.802441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.803154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:33.803461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.804865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.816218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 248/497] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:33.816872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.835882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 250/498] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:33.837002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:33.901881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:33.939063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc95860-8cc6-4d00-9dfe-ead58b74b5fa None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.945680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02074c-d28e-4795-96f7-db5297154675 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:33.973133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.973921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.974203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.974476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.980465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:33.981245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:33.981525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:33.981796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:33.982166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.983604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.992469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:33.994448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:33.998985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:33.999849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:34.000770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 249/499] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.001334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:34.013144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.013982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 251/500] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.015182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.032896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.104065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0433-418c-48b7-9cee-c643176dcb0f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.114811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c33a-f25a-419c-83fb-bf7090bb22bf None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.128921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.137198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.137985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.138264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.138536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.146605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.148057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.149511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.150291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.150907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.150907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.160855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4f6211e3-15d3-4c84-9640-3c75d0709313 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:34.162060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:34.162972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:34.163849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 250/501] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.165536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294a1a9-4625-4053-b06f-a8e3541a15f3 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.165818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294a1a9-4625-4053-b06f-a8e3541a15f3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.166096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2294a1a9-4625-4053-b06f-a8e3541a15f3 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.167490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 251/502] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:34.168052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294a1a9-4625-4053-b06f-a8e3541a15f3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:34.173002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:34.173857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 252/503] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.179420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.261720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.277321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2294a1a9-4625-4053-b06f-a8e3541a15f3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.279945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee367a-fd17-4c41-b73e-b4bd9276330d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.309724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.310388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.310666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.310936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.311759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:34.312064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.312811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.313468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.313554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c7bf2-122b-42a9-876f-0b373490950f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.313751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.314029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.314748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:34.315056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.316462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.328657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 253/504] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:34.329383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.349925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 252/505] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:34.351201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.351492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.351768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.362426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.368166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:34.370006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:34.427876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b98df6-4f52-4b8b-a7ea-277fad6c303c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.434741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 253/506] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:34.435809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.472372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.473031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.473308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.473574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.474391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:34.474692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.476148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.508886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.512228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 254/507] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:34.513109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.535000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6620edf-421e-42cd-ba5c-15f48afa4b45 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.568182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.568895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.569168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.569434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.570204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:34.570504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.572463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.580890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 254/508] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:34.581625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.609218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.618213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca758a01-1ff0-4c4c-bcb3-3cda5b49e33f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.653609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.654401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.654686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.654958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.662379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.663864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:34.679397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:34.680271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:34.681715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 255/509] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.682736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3163d-0a11-4388-9f74-529576c2f2be None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.682808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.716280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.716946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.717221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.717492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.720874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.783360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de12e9-f1a0-438c-a0e4-0efd23c4b557 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.783991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.785497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.816986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.817758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.818041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.818314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.828728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:34.830181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:34.847095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:34.848087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 256/510] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:34.849082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.853083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.896322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 255/511] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:34.897170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:34.929187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:34.949799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d002b-5fee-4cc7-96bd-599c3e32b408 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:34.982936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:34.983573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:34.983855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:34.984129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:34.995002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db65b83-d860-4534-adcd-8ce458793208 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.028651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.029428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.029700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.029968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.039860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.041318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.049831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.051304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.053875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:35.058909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:35.059767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:35.060815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:35.060815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:35.060815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.060815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:35.060815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 256/512] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:35.061134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.147513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.155198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 257/513] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:35.155841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.163275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae487b70-6e23-4871-bbe9-cecd5fb50fbe None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.197130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.197797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.198087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.198364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.199197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:35.199505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.200977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.217738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 257/514] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:35.218490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.258716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc331-12e4-4180-8b1a-040d8f5a94e4 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.261095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.292527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.293183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.293463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.293738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.294454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:35.294759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.296159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.307643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 258/515] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:35.308595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.315298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa5805-3452-4e50-bf60-d9bdf8046dea None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.347802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.348608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.348892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.349172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.359669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.361147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.367813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:35.379917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:35.380833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:35.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:35.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:35.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:35.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 258/516] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:35.381931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.409260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad602c8-12ee-401b-95db-4ee6968ccbe2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.442386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.443033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.443398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.443677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.444411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:35.444715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.446081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.457490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 259/517] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:35.458177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.483070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.487413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443c05a-9149-476d-be44-7f53b984e94e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.521002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.521818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.522098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.522371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.533481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.534951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:35.552656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:35.553697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 259/518] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:35.554724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.560736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c83a4-b6d3-48dd-955d-bd29ee8ae512 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.594306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.594947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.595226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.595495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.603516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.654712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce038f-b94d-4003-9dea-5d81569a3c8b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.661499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.662961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.686194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.688549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.689345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.689631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.690070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.690610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.692045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.711729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 260/519] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:35 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:35.713053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.778687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 260/520] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:35.778687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.838121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf349f8d-344c-400e-99fa-a6b09b9ece98 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.845690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.878243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.879234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.879516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.879844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.880719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:35.880969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.882363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.900322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 261/521] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:35.900322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:35.905340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520abe92-5ec8-4725-a8be-9e36b798d814 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:35.936754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:35.952982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:35.953792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:35.954073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:35.954343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:35.963162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:35.965553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:35.983077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:35.983962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:35.984878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 261/522] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:35.985559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.011705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca6dd8-f19b-4269-a7c9-d1c3a8a50666 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.046631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.047291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.047567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.047837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.048602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:36.048909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.050344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.062094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 262/523] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:36.062902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.092429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.105478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6e1d7-b10b-4b43-94c4-03da4f54124e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.142870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.143661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.143943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.144208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.155323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.156802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.168942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944256-9c2a-4194-aeb8-d2add0d636d0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:36.174338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:36.175190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 262/524] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:36.176707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.208669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.209435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.209661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.209925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.215471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.216998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.218450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:36.234121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:36.235013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:36.235843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 263/525] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:36.238899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.292931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef90c25-f6fa-4c93-9a7b-0c5140327082 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.327471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.330685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.331540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.331857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.332164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.333155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:36.333363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.334811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.343623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ba581-a3cd-4823-9182-f6dccc2f5f66 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.357005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 263/526] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:36.357005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.384990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.385786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.386058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.386329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.393200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.394690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:36.408528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:36.409368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:36.410612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 264/527] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:36.415334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.456473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.464970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647e9a5-c4cb-40e9-9658-f04887b9827f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.500736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.501401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.501685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.501961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.502685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:36.502997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.504414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.517297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 264/528] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:36.521206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.523828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e5127-afb7-4680-9159-da49bebe7432 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.559024 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.559682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.559958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.560231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.561082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:36.561386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.562813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.581341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 265/529] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:36.582186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.593892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.640173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2a3e2-7fee-4a55-8e27-a0528e59668d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.674557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.675205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.675550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.675825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.688716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffc048-e1e5-470f-8565-b6d80fdcb65c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.702003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.723864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.724545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.724823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.725107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.725870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:36.726180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.727597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.745838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 266/530] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:36.745838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.758424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.759931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.812329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:36.851216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9ff3-c28b-45f6-b8c8-61f789f06500 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:36.886077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 265/531] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:36.886956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.893364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:36.894170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:36.894466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:36.894736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:36.902188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:36.903640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:36.918123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:36.918999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:36.919839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 267/532] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:36.920953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:36.963461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:37.009712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a5e1c-50ac-441d-899d-9afc730f6d6f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.031886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bcba7-dab8-4803-aeef-1a67030c14c7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.043773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.044452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.044736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.045007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.045825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:37.046131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.047568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.052871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 266/533] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:37.053638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.075361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:37.089948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.091007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.091545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.092057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.093240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=847292d322f6483ba5ff02294ad409dd) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:37.093774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.095673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.103523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 268/534] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:36 2022] GET /identity/v3/role_assignments?user.id=847292d322f6483ba5ff02294ad409dd&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:37.105148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.225881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768490a-1141-4a88-b882-a612b01a2d82 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.246377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d843a2-53bb-416f-b6cb-d2c78bec6d8e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.254711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:37.289849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.291443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.291733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.292010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.298021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.299046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.299559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.300032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.301202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:37.301727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.303415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.317008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.318850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.323705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 269/535] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:37.327821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:37.354077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:37.355237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:37.356135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:37.356135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:37.356135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:37.356135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:37.360863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 267/536] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:37.365095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.429995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:37.517007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772757e-f5a8-4c15-86d8-c47c41d206db None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.562240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4ac45-bff0-4626-b3b6-5662f3f6f17b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.585947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.586615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.586893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.587739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.588613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=94ce1868b4ac4d2a8273867aab1bfe80) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:37.588983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.590442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.605408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 270/537] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/role_assignments?user.id=94ce1868b4ac4d2a8273867aab1bfe80&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:37.609648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.610651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.611138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.611610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.612609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:37.613169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.614852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.628210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 268/538] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:37.635271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.786788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67377640-01fd-452f-b005-315f909a26a2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:37.808804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:37.842240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:37.842963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:37.843246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:37.843520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:37.850663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:37.852129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:37.866809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:37.867714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:37.869409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 271/539] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:37.874225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:37.932761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:38.021566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80054c-82c8-4a4f-9a3d-a9e9f875bf5e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:38.076761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:38.077443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.077726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.078003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.078798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:38.079108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:38.080607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386820c-8f19-4c0c-b9e0-a3dbf1f6481c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:38.111041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 269/540] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:38.116574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:38.141807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046213-8a6f-436e-8450-fb52a3af3325 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.142121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046213-8a6f-436e-8450-fb52a3af3325 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.142377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac046213-8a6f-436e-8450-fb52a3af3325 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.144030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 270/541] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:38.148575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.148862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.149139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.180242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:38.182035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:38.237137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2cfe9-f60f-45c5-9c40-b53e2416df71 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:38.283758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:38.284971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.285458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.286009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.287033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=6c28d40778004a2d8f2b8e2cb38a1f12) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:38.287524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:38.289431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:38.306459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 272/542] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:38 2022] GET /identity/v3/role_assignments?user.id=6c28d40778004a2d8f2b8e2cb38a1f12&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:38.311429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 271/543] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:38.362344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:38.416936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:38.500584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b7d1b-bd42-43a9-ab2c-a67266a5340f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:38.546280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:38.547096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.547361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.547634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.562243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:38.569939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:38.571373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:38.573902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 273/544] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:38.578133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:38.616576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:38.732449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64251636-f5bf-4c38-9af7-1169e95e5565 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:38.773344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:38.774022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.774297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.774736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.778050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:38.778368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:38.779946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:38.792341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 272/545] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:38.797328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:38.920374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:38.925783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d650efe-7ec4-4f53-b4c0-7fc88f57be76 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:38.974497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:38.975291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:38.975568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:38.975822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:38.990431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:38.991910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:39.015533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:39.016437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:39.017272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:39.017272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:39.017272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:39.017272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:39.017272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 274/546] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:39.022516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:39.148285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:39.171091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f738c49-6221-424d-9bb1-2a626c505572 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:39.232010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:39.233088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.233576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.234055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.234984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:39.235515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:39.237129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:39.258497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 273/547] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:39.264553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:39.410101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ec005-9f6c-458a-a665-1b05d656de42 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:39.451854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:39.453009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.453498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.453992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.468724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:39.470494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-b37c2174-1b2b-4529-8ac6-65cd1dad0636 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:39.494724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:39.495636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:39.497124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:39.499803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 275/548] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:39.505010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:39.576026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e9a8-e01f-4f3e-a194-433d01edb4be None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.576371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e9a8-e01f-4f3e-a194-433d01edb4be None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.576611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81e9a8-e01f-4f3e-a194-433d01edb4be None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.580321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 276/549] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:39.583006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.583294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.583566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.600728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:39.601908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:39.620653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:39.662639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d8b07-45fe-417f-b698-87235bbc09f8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:39.713614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 277/550] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:39.721574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:39.722501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.723012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.723492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.724562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:39.725080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:39.726748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:39.736171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:39.751762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:39.763296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 274/551] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:06:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:39.769215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:39.913381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097c5a9-79e6-4ab0-9398-7c2fe82ada43 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:39.956020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1da4c0-1e42-4b7e-a0ed-340aba87c364 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:39.978291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:39.979093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:39.979616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:39.979895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:39.996652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.001334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:40.010257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:40.011226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:40.012142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 278/552] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:40.020573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.030805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.031445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.031722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.031989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.149810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.156153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.157667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b0130-f489-4856-aa9d-2f8bfdcfef6d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.204567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48976e4d-8026-4a73-960d-f7d65c15cba7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.241278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.242278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.242754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.243694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.244626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:40.245077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.246784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079286c2-76e5-4200-8da1-818050f1a5b8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.255815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 279/553] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:40.257428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c783-3bee-456b-a13e-18fc2a077c56 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.257861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c783-3bee-456b-a13e-18fc2a077c56 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.258298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c783-3bee-456b-a13e-18fc2a077c56 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.260252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 280/554] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:40.265155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c783-3bee-456b-a13e-18fc2a077c56 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.289966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 275/555] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:40.296026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.296889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.297317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.316600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:40.318035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:40.335968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.380105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c783-3bee-456b-a13e-18fc2a077c56 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.392912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 276/556] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:40.400327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.425372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.440826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.441884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.442315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.442861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.456662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.458467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:40.477085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:40.478709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:40.479855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:40.479855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:40.479855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:40.479855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:40.481864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 281/557] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:40.482805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.500786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad413e4-af5a-4cb6-a7c5-f23f869f4f6c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.536926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.537822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.538102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.538494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.549570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.551029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-cbd08db1-4ee4-4c32-aba8-a007ff5db5b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:40.570087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:40.571469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 277/558] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:40.572777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d048-5ebc-41d3-969b-2a06e016a2aa None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.573126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d048-5ebc-41d3-969b-2a06e016a2aa None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.573196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0d048-5ebc-41d3-969b-2a06e016a2aa None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.574502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 278/559] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:40.576410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d048-5ebc-41d3-969b-2a06e016a2aa None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.594347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba169bbb-b970-4af6-9ea6-07f558021eb9 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.609967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.632061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.633006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.633286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.633558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.645571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:40.653949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:40.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 282/560] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:40.661175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.668704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.696685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0d048-5ebc-41d3-969b-2a06e016a2aa None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.734362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.735343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.735770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.736051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.736810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:40.737123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.738532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65ccdc-0d8f-4a46-91a0-ea3a2d89d6e4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.755313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 279/561] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:40.755313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.755313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.755313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.769459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:40.770614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:40.783640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0da52-a5ed-4821-9d93-dad81070be04 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.819732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.820718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.821158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.821433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.822159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:40.822469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.823894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.840165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 283/562] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:40.840165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.847209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 280/563] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:40.848027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:40.876032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.922982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:40.940287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fc528-3c9b-458e-87e4-59852a0a2553 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.951316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07628ec2-c6c6-4329-9701-50d731689be9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:40.977347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.978573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.978851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.979125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.980374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:40.980684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.982056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1476c5f4-bfe8-4b4f-880b-07e06146fa39 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.986121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:40.986915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.987193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.987470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.992003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 284/564] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:40.994053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ddd4c-cd3b-48e8-975f-2c3615d28fde None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:40.994280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ddd4c-cd3b-48e8-975f-2c3615d28fde None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:40.994558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ddd4c-cd3b-48e8-975f-2c3615d28fde None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:40.994712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:40.996280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:40.997017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 285/565] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:40.997731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ddd4c-cd3b-48e8-975f-2c3615d28fde None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:41.011144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.012094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 281/566] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.013067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.097787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.105280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ddd4c-cd3b-48e8-975f-2c3615d28fde None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.112396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5175d1-32d3-4d6f-90d0-f33baab4c20c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.137786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.138563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.138843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.139114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.141174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.145663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.146412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.147121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.147236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.147462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.147733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.158477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5785ebb5-bde1-47d6-acef-453279b227cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:41.161622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.162639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:41.163710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:41.166175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.167309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 282/568] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.168487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.169668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 286/568] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.170943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.171233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.171509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.189168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:41.190360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:41.253998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 287/569] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:41.254801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.276927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f779c28-e719-407e-b430-7787da3b01dc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.312618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.313410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.314494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.314770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.326425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.327885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.337883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:41.345256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.346362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 283/570] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.347613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.357513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c66463-aa25-4a51-bae4-d067b97dbee3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.390541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.391201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.391477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.391744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.392588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:41.392896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.393025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.394332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.411404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 288/571] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:41.412014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.452284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863250-92eb-471b-a5f7-07535b99a487 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.484768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.485410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.485685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.485953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.486727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:41.487032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.488482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.500370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 284/572] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:41.501140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.508186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468bb4f9-2a9b-4f17-8740-07c06b38a737 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.543047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.543654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.543939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.544217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.545018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:41.545326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.546688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.554433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 289/573] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:41.555208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.565523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.610767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eebe70-a244-4f50-b2f8-2b37ccb1702e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.613163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.645255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.646044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.646324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.646602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.656592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:41.657468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21c586-493d-4bcf-a6c1-c746c5ba0fe2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:41.664253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.665229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:41.666036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 285/574] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.666697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.693684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.694341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.694618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.694887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.695596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:41.695925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.697870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.709089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 290/575] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:41.709857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.770369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5501985-9a30-45cd-bafe-693fee040e90 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.777802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.803836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.804574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.804850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.805116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.812202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9894ad-0214-4a15-98b9-9e5cde1980d6 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.815878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.817340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:41.834316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:41.835346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 286/576] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:41.836627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.848443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.849105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.849408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.849656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:41.850481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:41.850791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:41.852258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:41.855486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:41.858731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 291/577] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/role_assignments?user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:41.859483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:41.949465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d9a80-3e4a-498b-ac89-599d840ae6cc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.973423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bcef41-41f1-4bfc-95df-bbdc257feae5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:41.987882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:41.988838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:41.989185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:41.989522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.002270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.003777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.007583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.014707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.015387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.015665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.015938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.016723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.017028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.018406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:42.023439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.025203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.026290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:42.026290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:42.026290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.026290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:42.030416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 287/578] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.035520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 292/579] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.035654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.035758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.112168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.148675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440b45e-f7e1-4f97-9056-d3bddc795828 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.151028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10be9f-e6df-482a-9af8-02873f86d497 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.184600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.185395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.185672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.185952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.186663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.186968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.188594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.192890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.193681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.193961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.194234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.202030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.203514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.205713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 293/580] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.206719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:42.220221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.221198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 288/581] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.222969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.266864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.319419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112650b9-e66c-4a74-a261-3b7c3e313645 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.335269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fff797-be70-49c9-97c2-4b174da57374 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.357880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.358560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.358835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.359109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.359937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.360249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.361643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.364618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.373516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.378441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 294/582] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.378588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.378588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.378588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.378819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.390136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.391606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:42.409098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.409960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.410842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:42.410842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:42.410842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.410842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:42.410842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 289/583] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.411114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.486251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906deda-e74d-412f-adea-f630a8c1d8e1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.499864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.520899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b9b70-0837-49b2-a303-97cfecb3204a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.522453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.524019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.524349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.524631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.532660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.534099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:42.548904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 295/584] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.550865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.560963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.561660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.562216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.562216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.563033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.563343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.564862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.588699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 290/585] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.588699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.630553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.656836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3c1fa4-3503-44ef-8113-1ed3bdda11b0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.692804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.693596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.693873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.694139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.695304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1787cd-1878-469d-9907-3d8d5d19b8ac None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.701145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.702590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:42.717806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.718743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:42.719701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:42.723542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 296/586] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.724371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.733610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.734268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.734550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.734939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.735659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.735967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.737502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.754523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 291/587] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.754523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.762676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.834681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5bd36-ec70-4307-aaba-da95cff0748e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.853168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.857358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828a6604-afd3-4dfd-9746-5532818de9e2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:42.869531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.870252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.870534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.870808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.871591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:42.871896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.873362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.884626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 297/588] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:42.885417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.900149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:42.901331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:42.901793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:42.902387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:42.914120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:42.916282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:42.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:42.935453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:42.936588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:42.936588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:42.936588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:42.936588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:42.938391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 292/589] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:42.939318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:42.986407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:42.996617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccc5dc-0d7b-43af-8873-61e5a6df4a5a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.032613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.033291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.033573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.033855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.034685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:43.034994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.036453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.047767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b90274-7889-43cf-a037-89fe2c3798f2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.058444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 298/590] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:43.058444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.092043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.092973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.093253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.093526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.103667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.105478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:43.121148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:43.122077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:43.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 293/591] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:43.123580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.136656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.167003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b9f10e-a249-4f01-9a67-7bc8a347c806 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.204239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.205274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.205675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.205947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/847292d322f6483ba5ff02294ad409dd/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.220908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=847292d322f6483ba5ff02294ad409dd, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.222421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.229670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00136975-4ea6-4a3a-a87c-14695028a465 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.234034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.248403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 299/592] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:43 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/847292d322f6483ba5ff02294ad409dd/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:43.250348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.276452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.278423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.278932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.279673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.280671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:43.281478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.283970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.303145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 294/593] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:43.304778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.388231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3785af0-2f51-43f9-9265-10eb7ee204f3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.391471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.429187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c9c7d0-bd58-4c44-8585-2985d0a246cb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.433206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.434279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.434782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.435278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.447251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.449034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.467288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.467969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.468250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.468553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.469394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:43.469703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.471151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:43.473253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:43.474103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:43.474966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:43.474966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:43.474966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:43.474966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:43.478192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 300/594] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:43.482945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.495571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 295/595] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:43.500335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.545054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.596830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6832d-4021-4b1d-8778-98797cf66606 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.628033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb5549-77aa-42db-8e44-e005373987ba None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.634559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.635557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.635987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.636439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.644213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.645879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:43.661527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:43.662416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:43.663296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:43.663731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.666370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 301/596] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:43.667355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.671150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.671846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.673086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.673745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.674783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:43.675289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.677051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.696388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 296/597] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:43.697484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.798219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd12cde3-e42f-44be-8362-b85ab6de34d4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.816724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.822489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76005b5a-5177-45fe-acfc-f746b5bbcc8e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.835212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.835979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.836255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.836550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.848228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.849644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.859635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:43.860589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:43.861075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:43.861752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:43.862745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:43.863261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:43.864932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:43.867758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:43.869174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:43.870047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:43.870047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:43.870047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:43.870047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:43.870047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 302/598] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:43.870760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.881365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 297/599] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:43.882581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:43.940644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:43.986427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e92c3-9ac3-4ed0-9910-07a8cdb84ae1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:43.997715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33191b7-9de5-4375-84dd-afe2724060cc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.022804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.023589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.023864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.024132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.031565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.033196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.037367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.038297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.038782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.039259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.040245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:44.040799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.042376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:44.047948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:44.048830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:44.049712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 303/600] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:44.050631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.058516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 298/601] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:44.059739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.078848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.162651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e21491-3114-46c5-a8b3-e91c5c2bd46b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.174598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009bf930-904b-451a-a998-7d8efe89e4c8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.184089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.202518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.203351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.203635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.203923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.211940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.212632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.212921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.213189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.213976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:44.214286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.215731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.220218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=aa1fb96699054e5c94fcc188fd9535be, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.221830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d6f4e-97c7-487f-9358-7d4c41220990 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.228855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 299/602] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:44.230007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 304/603] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:44 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b432-09ff-40cd-b9df-51857b9913e7 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b432-09ff-40cd-b9df-51857b9913e7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b432-09ff-40cd-b9df-51857b9913e7 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 305/604] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:44.245703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3b432-09ff-40cd-b9df-51857b9913e7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.341284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.379744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f385fb-9d1d-4bef-8c56-1ff532083a27 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.385970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3b432-09ff-40cd-b9df-51857b9913e7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.422892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.423854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.424229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.424646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.432974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.433183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.433878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.434161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.434825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.435246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/94ce1868b4ac4d2a8273867aab1bfe80/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-db034448-1dd4-4522-be85-51e84c3fb5c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:44.450093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:44.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:44.452028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 300/605] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:44.453526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.453526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.453526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.454521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=94ce1868b4ac4d2a8273867aab1bfe80, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.455876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.475427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:44.477013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:44.478344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 306/606] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:44 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/94ce1868b4ac4d2a8273867aab1bfe80/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:44.479243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.505850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.558202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 301/607] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:44.596879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.597615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856979e-8599-4c74-9c01-2a1ec0ee7095 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.624408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.643826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.645697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.646364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.647473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.648631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:44.650828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.652536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.680031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 307/608] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:44.687148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.789575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e03d7f-63e4-4192-a57c-4dd03436bb87 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.840313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.841122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.841400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.841670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.846105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6285b309-88e7-40b6-bd01-d536cae9b7ef None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:44.855506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:44.863886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:44.864943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:44.865785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 302/609] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:44.873291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:44.891617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:44.892541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:44.892818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:44.893088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/6c28d40778004a2d8f2b8e2cb38a1f12/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:44.914806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:44.920520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=6c28d40778004a2d8f2b8e2cb38a1f12, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:44.922253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:44.947314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 308/610] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:44 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/6c28d40778004a2d8f2b8e2cb38a1f12/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:45.041527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19555c09-8c4b-4317-8233-e0b5741f5691 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:45.092868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:45.093933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.094416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.094873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.095917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:45.096443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:45.098127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:45.112907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 303/611] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:45.119542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:45.168694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:45.247482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ccaa-7a52-4b65-ba54-9fd1e7c967cc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:45.284968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:45.285922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.286412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.286889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.299187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:45.300614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:45.320017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:45.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:45.321930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:45.321930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:45.321930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:45.321930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:45.325956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 309/612] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:45.327306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:45.442051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:45.476625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f2e6cd-5267-4b33-a3b8-ca688b6c77c8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:45.530916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:45.531685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.532116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.532614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.533888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:45.534323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:45.535748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:45.553766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 304/613] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:45.560101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:45.661152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:45.677662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4675d-42eb-4c9b-a0b9-1320079cb8dd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:45.715786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:45.716619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.716910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.717190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.726106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:45.728152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-3477fd21-1470-4c69-b06c-a74434069643 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:45.745433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:45.747589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:45.749900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:45.756350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 310/614] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:45.756571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:45.884117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44411ef-fb11-4040-a68a-f453b1b7b54a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:45.900550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a185b2-e1b8-4dba-9c89-d0b69fcb87c2 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.900829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a185b2-e1b8-4dba-9c89-d0b69fcb87c2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.901094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a185b2-e1b8-4dba-9c89-d0b69fcb87c2 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.902682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 311/615] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:45.907524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.907804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.908080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.927047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:45.928230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:45.944987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:45.945672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:45.945953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:45.946267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:45.947111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:45.947420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:45.948896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:45.958560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:45.970815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 305/616] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:06:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:45.978408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.029366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 312/617] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:46.065145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.172001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:46.183663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d4b599-8f25-4e36-84e9-e86cc53a7826 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.204061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70afd21-63ba-437a-a66f-2b6e3c63ab04 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.247269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.248403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.248873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.249351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.260134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.261252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.261756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.262237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.273056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:46.281130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:46.282011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:46.282915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:46.283787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:46.284684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.286832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 313/618] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:46.292531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:46.325009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:46.325969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:46.327307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:46.327307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:46.327307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:46.327307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:46.327307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 306/619] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:46.333209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.406383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d60c6-2c6e-4899-8afa-8e986d3be906 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.432906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:46.460813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.462438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.462954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.463436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.464429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:46.464958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.466680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.472711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:46.476252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 314/620] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:46.481433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.562423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f097fa1-de58-4159-91f4-50fd97e56014 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.632505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.634064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.634579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.635065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.637095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:46.637616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.639938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.668836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 307/621] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:46.674111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.690502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00658d92-49b3-4a25-beb8-695a68c52eba None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.756361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.757886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.760631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.761141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.773303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.775570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:46.794976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:46.795927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:46.796861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:46.796861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:46.796861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:46.796861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:46.799186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 315/622] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:46.804197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.860705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:46.907319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130aece-087e-47a0-9f15-90809091e561 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.911717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79886168-47b2-45c8-9357-753924b0bfc7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:46.920737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:46.948756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.949617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.950079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.950474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.951506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:46.952004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.953556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:46.953856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f99f-5374-4522-953c-7fd5a2950eec None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.954684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.955117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.955822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.964216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:46.966678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:46.974453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 316/623] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:46.974453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faea78-fa21-43fd-987a-90c8dcef1375 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:46.974453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faea78-fa21-43fd-987a-90c8dcef1375 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:46.974453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4faea78-fa21-43fd-987a-90c8dcef1375 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:46.974453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 317/624] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:46.975457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faea78-fa21-43fd-987a-90c8dcef1375 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-c3cf4e17-9e1d-402c-bdbb-6305e5a5fccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:46.996028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:46.997194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:46.998271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:47.000529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 308/625] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:47.002781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.003228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.003643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.024056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:47.025452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:47.093573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4faea78-fa21-43fd-987a-90c8dcef1375 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.106168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.116456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 309/626] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:47.117360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.149546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.150357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.150636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.150909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.159817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.161545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:47.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:47.179682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 15:06:47.181310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:47.184627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 318/627] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:47.185504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.239013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.241957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c390977-c3de-4d77-b881-42dc214844d3 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.279887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.280836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.281215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.281557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.282564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:47.282942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.284513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.298350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33e646-1154-4343-95ae-4da3d1c05b21 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.298594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 310/628] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:47.299578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.336005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.336872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.337153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.337427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.347971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:47.356510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:47.357414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:47.358234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 319/629] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:47.359821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.388517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.418930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98775d7b-9f1c-4c27-8140-da5cd68e4afc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.453335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.454033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.454309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.454586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.455440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:47.455746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.457277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da83fa36-59d3-48bd-9b19-36932d719e62 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.471314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a38d1-900f-4768-a23a-a4eae32d9862 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 311/630] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b697fe-d80e-49dd-9a3e-214f9d319140 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b697fe-d80e-49dd-9a3e-214f9d319140 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b697fe-d80e-49dd-9a3e-214f9d319140 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 312/631] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:47.486107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b697fe-d80e-49dd-9a3e-214f9d319140 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.503872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.513036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.514584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.515313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.515751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/ec47b745ff5a4ced9b47ed5ecd2b28cb/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.534856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=ec47b745ff5a4ced9b47ed5ecd2b28cb, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.536937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.553300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 320/632] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:06:47 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/ec47b745ff5a4ced9b47ed5ecd2b28cb/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:06:47.554964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.599026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b697fe-d80e-49dd-9a3e-214f9d319140 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.639609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.640339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.640624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.640904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.641732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:47.642046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.643532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde6dba-672c-4ea8-8351-2686c3367859 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.653094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 313/633] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:47.654459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ff15-dfde-44eb-a3f2-d78e22d47f6a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.654753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ff15-dfde-44eb-a3f2-d78e22d47f6a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.655090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099ff15-dfde-44eb-a3f2-d78e22d47f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.674384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.674634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2099ff15-dfde-44eb-a3f2-d78e22d47f6a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:47.675654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2099ff15-dfde-44eb-a3f2-d78e22d47f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:47.700326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672658ed-79ac-40bb-8cb6-daede7f34f19 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.742637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.743871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.744268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.746344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.750982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 314/634] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:47.752336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2a82f-3e58-4de7-8a61-2d9648fd6cad None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.752629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2a82f-3e58-4de7-8a61-2d9648fd6cad None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.752903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2a82f-3e58-4de7-8a61-2d9648fd6cad None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.754376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 315/635] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:47.754961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2a82f-3e58-4de7-8a61-2d9648fd6cad None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.767319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.825779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.827679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.868257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2a82f-3e58-4de7-8a61-2d9648fd6cad None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:47.901954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:47.902750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.903029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.903303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.914079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:47.915581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:47.917138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-6f6fee37-a40c-4443-8d30-717580638e5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:47.933152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:47.934101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:47.934999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:47.934999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:47.934999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:47.934999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:47.934999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 316/636] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:47.935994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:47.936359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:47.936705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:47.943762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 321/637] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:47.944988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:47.954794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:47.955964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:48.012459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.024865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 317/638] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:48.024865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.071719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467da18-0af9-4b0c-95c6-fe9a86a70f06 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.108884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.109678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.110069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.110343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.120767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:48.129163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:48.130111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:48.130950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 322/639] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:48.132095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.143144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cca258-1014-4dd3-9a0d-f02e5205aed7 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.179149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.179812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.180091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.180887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.181656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:48.181961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.183336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.183446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.194641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 318/640] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:48.195476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.249539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.255277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b118609-f4f4-4ecf-ba34-09cc0af261c8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.291528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.292236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.292647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.292919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.296155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6379a4-6ef6-44ed-a811-688a2c58706f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.304909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.307103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:48.325534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:48.326614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:48.327738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:48.327738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:48.327738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:48.327738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:48.327738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 323/641] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:48.328090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.335603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.336426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.336747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.337020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.347323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:48.355165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:48.356289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 319/642] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:48.362169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.430102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.441599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b6f5a-7f35-41b0-ab62-ddbe33a00029 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.470324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfec053-6759-42ad-bbf3-e0d13e9962f3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.478349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.479609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.479899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.480176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.481077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:48.481391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.482768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.483552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.497260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 324/643] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:48.497260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.510805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.511667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.511947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.512214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.519780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.521291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:48.535913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:48.538381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 320/644] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:48.540893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.609874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91fc847-123c-4b36-982b-168060330dbe None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.645182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.645389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1f88f-94a0-403a-9945-1c76266a6c47 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.645850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.646134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.646413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.647136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:48.647446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.648873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.665883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 325/645] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:48.665883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.681072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.689140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.689864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.690096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.690369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.691124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:48.691431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.692863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.701966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 321/646] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:48.702910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.740708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.774085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd8ffc-1ddf-475c-ae53-21fac14c8f23 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.807713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.808536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.808816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.809090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.818743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce853fcd-ce55-4a29-82fd-fd89444ba86b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.818987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.820477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:48.839273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:48.840353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 326/647] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:48.841501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.861984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.862586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.862873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.863149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.863985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:48.864316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.865897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.889336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 322/648] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:48.889336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:48.911851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.947462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ceecde-2256-4d69-8ba3-796f01a75f63 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:48.983039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:48.983832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:48.984114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:48.984417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:48.986428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:48.991594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:48.993568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:48.997994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa98e0-25c7-4815-a3ce-d5a7e94f452b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:49.010025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:49.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:49.011786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:49.016032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 327/649] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:49.016909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.037406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.038200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.038488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.038767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.049901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.051364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:49.070299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:49.071959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 323/650] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:49.072879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.135481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f6e3a-0435-4435-a908-f3b4baf281f1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.161445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.172219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.172847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.173132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.173418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.174145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:49.174455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.175827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.181961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3ac61-a37d-4a91-b34b-6d8e880e5994 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.189637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 328/651] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:49.194414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.222667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.223320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.223604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.223878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.239500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.296078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076a511-a53c-45e3-9b8c-5d80b3c1c0a2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.299262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.300774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.332030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.332737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.333023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.333305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.334093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:49.334406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.335832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.351909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 329/652] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:49.351909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.401346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.412373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 324/653] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:49.413153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.463551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.464261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd7f68-4265-410f-bda0-d17fe4357d2f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.500322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.500997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.501289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.501569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.502294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:49.502638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.504015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.517211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 330/654] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:49.518015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.527269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd452f-f410-41fd-979a-8e6bbfab4709 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.562032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.563067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.563490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.563914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.571957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.573519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:49.589969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:49.590988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:49.591931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 325/655] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:49.592579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.632023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bea066-00f6-4340-91fa-aef0c7e0cd7e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.640326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.669583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.670171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.670453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.670726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.671554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=847292d322f6483ba5ff02294ad409dd) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:49.671866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.673339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.679293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 331/656] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/role_assignments?user.id=847292d322f6483ba5ff02294ad409dd&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:49.680083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.721988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3503d-cd67-43e0-a926-a12bdf6ec8a1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.725878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.771644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.773858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.775323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.775815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.790977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.792536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:49.812380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:49.813348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:49.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:49.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:49.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:49.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:49.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 326/657] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:49.814496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.826270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1397d7-d695-4dca-8d80-39c955aac467 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.866177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.867097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.867482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.867765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.876819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.878352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d90e8c76-40cb-497b-9c7f-e12f83383f80 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:49.893412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:49.894548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:49.895511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 332/658] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:49.896122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071260-9781-413c-9f20-dc4aefe2acce None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.896525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071260-9781-413c-9f20-dc4aefe2acce None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.896801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef071260-9781-413c-9f20-dc4aefe2acce None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.898268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 333/659] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:49.898969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071260-9781-413c-9f20-dc4aefe2acce None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:49.931575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c183-7a7a-44e5-963d-63e8c972686c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:49.936134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:49.970604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:49.971437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:49.972025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:49.972350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:49.973371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:49.973681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:49.975126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:49.992393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 327/660] 10.222.0.45 () {56 vars in 1137 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:49.993709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.010429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.020129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef071260-9781-413c-9f20-dc4aefe2acce None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.056485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.057383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.057662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.057939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.058665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:50.058980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.060361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d7a56-24be-4b7a-9d79-550c8b7c67da None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.082061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 334/661] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:50.082061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.082061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.082061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.095263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:50.096470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:50.120000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75317b-35fb-4c4b-bf52-9638896241d9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.157383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.158056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.158440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.158713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.159656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:50.160056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.161545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.168469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 335/662] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:50.168469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.188951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 328/663] 10.222.0.45 () {56 vars in 1139 bytes} [Tue May 24 15:06:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:50.190028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.190951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.263781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.277322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5eb17-6a9d-4b62-87f8-9009b7227e0b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.295374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549043-6678-4a30-b499-fb26f3e61020 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.316234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.316934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.317217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.317485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.330724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.331512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.331790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.332060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.339642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.341119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:50.354790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:50.355703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:50.356615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 329/664] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:50.357611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.393478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.395764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.418341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.463263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c033762-403f-45bf-bb75-9f795132ecf2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.494830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.500743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.501391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.501670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.501944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.511006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 336/665] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:50.512089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.576109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.577617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.624692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247654-06de-4404-8e45-f5bb5c05d2e7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.646610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.667457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.668248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.668557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.668826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.679741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:50.686849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 330/666] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:50.686849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:50.688055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:50.688999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:50.689856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 337/667] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:50.691960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.725171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.798418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3d5c3-9ed8-4ef1-a3fd-5f3e0a548990 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.801956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f50ff6-b6bd-4f41-a271-43e10d7544b7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.835077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.835916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.836141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.836439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.838947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:50.839672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:50.839958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:50.840234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:50.841065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:50.841374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.842822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.847494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:50.848947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:50.860059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 331/668] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:50.860059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:50.873112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:50.874088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:50.874980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:50.874980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:50.874980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:50.874980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:50.874980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 338/669] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:50.875293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:50.908337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.962488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:50.973117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b1fe6-1ae4-42d5-bec4-deef51735f30 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:50.984323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9471d-be32-408e-a358-4dbf0eaf88ca None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.019152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.019819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.020095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.020396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.021136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.021439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.021746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.022537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.022801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.022895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.023072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.035229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 339/670] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.035229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.037830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.039407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:51.058755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:51.059638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 332/671] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:51.060933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.138421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.146544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810b86c-e023-4ccf-9761-16acf45c911b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.170569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa2ba-74e3-444b-aa4b-e07b4396df89 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.180281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.181426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.181698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.181962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.182816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.183124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.184580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.196754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 340/672] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.198161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.214437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.215814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.216409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.216917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.218498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.219056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.221190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.244796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 333/673] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.244796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.250331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.324340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d206ad-819e-45ab-9f4a-060b3c1d7c9c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.362271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.363085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.363360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.363631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.376348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.377830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.391009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4eced-7a1a-4c0c-9630-cff7bf6e3b23 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:51.399863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:51.400805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:51.401729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:51.401729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:51.401729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:51.401729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:51.404153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 341/674] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:51.405287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.419287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.439623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.440288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.440619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.440892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.441611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.441910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.443297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.456716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 334/675] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.457486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.534443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f5e9c-a897-4654-b14c-0cdf7be3d0d0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.538344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.579629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.580509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.580798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.581073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.588854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.589086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c4021f-b2db-4488-a360-362d4df5b5d1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.590357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:51.605707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:51.606653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:51.607959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 342/676] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:51.608813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.631226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.631892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.632172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.632497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.634009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.634318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.636114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.652242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 335/677] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.653100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.691168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.727073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc781ac-8cdc-452f-9027-c1c8daae63ba None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.755141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5702bb-cc5b-4d8e-9e3b-c7abffbed9e2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.765358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.766519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.767004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.767586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.776272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.778462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.782043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.795085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.795768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.796118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.796504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.797366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.797745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:51.798704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:51.799550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:51.800572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:51.801008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.803541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 343/678] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:51.804759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.824117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 336/679] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:51.824117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:51.921541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf135244-60ad-4b60-b715-d1d1d39a379f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.950301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be22f77-a3db-4d9f-b248-7cc4c28dc3d0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:51.953659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:51.960445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.961609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.962911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.963410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.976589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.978960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:51.994188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:51.994848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:51.995131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:51.995409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:51.996256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:51.996599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:51.996849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:51.997912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:51.999117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:51.999791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819cfb2-4b09-481d-abbf-64d90831e40c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.000885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 344/680] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:52.004344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.012414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 337/681] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:52.013702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd011f0-9d40-4859-bfe2-1fd79ba0946c None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.013992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd011f0-9d40-4859-bfe2-1fd79ba0946c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.014269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd011f0-9d40-4859-bfe2-1fd79ba0946c None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.016417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 338/682] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:52.017530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd011f0-9d40-4859-bfe2-1fd79ba0946c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.067732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.122862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672d3284-e104-4d06-b170-61a9640058e5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.131256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd011f0-9d40-4859-bfe2-1fd79ba0946c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.159206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.160069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.160382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.160658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.161807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=94ce1868b4ac4d2a8273867aab1bfe80) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:52.162174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.163620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.168931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.169712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.170095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.170470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.171406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:52.171819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.173411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0bbf12-8794-48b1-87b2-a58760af00a1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.177343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 345/683] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/role_assignments?user.id=94ce1868b4ac4d2a8273867aab1bfe80&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:52.177603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.200745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 339/684] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:52.202453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.202849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.203230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.221609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:52.223015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:52.256106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.316754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 340/685] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:52.318007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.328528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3bea4-1a25-48b1-a30a-a3983d0dc625 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.373156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.392837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.394015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.394969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.395400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.396576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=6c28d40778004a2d8f2b8e2cb38a1f12) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:52.397040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.398874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.407272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 346/686] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/role_assignments?user.id=6c28d40778004a2d8f2b8e2cb38a1f12&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:52.409497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.467971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c89819-f5a3-495e-9daa-64caad24b146 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.514028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.514948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.515408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.516914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.552321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b147468-7190-4023-b73c-514b92bdb422 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.573173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.593758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.594577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.594859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.595132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.608334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.609994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:52.628937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:52.629868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:52.630770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:52.630770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:52.630770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:52.630770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:52.630770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 347/687] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:52.636519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.636686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.638150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.701387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.762084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fd0ce-66e7-4477-818d-de896ab423f8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.791644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 341/688] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:52.797476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.804110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.804812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.805090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.805896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.806621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:52.806922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.809091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:52.825012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 348/689] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:52.833248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:52.876494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.932456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fa5a6-c7b5-4559-af31-216737471b72 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.964436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550854f-b990-4f68-ba99-09c536152854 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:52.975161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:52.982062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:52.983230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:52.983606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:52.983960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:52.998083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:52.999579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:53.019194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:53.020132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:53.021550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:53.021550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:53.021550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:53.021550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:53.021550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 342/690] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:53.026038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:53.038621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:53.039964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.040639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.041131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.042035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:53.043675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.070186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 349/691] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:06:52 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:53.183608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a9acd-05f6-421f-b077-02c8317ba4ac None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:53.221098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:53.221770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.222052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.222321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.223052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:53.223366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:53.224868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.238790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 343/692] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:53.245053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:53.368753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:53.386949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abccbfa4-6bef-4c7a-af4a-bc55bbad1a1c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:53.422733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:53.423538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.423820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.424092 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.431708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:53.433197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:53.447657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:53.448673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:53.450166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 350/693] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:53.454754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:53.504608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:53.561263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c66cd-dc5a-4b9a-8362-5a1b5b3015d1 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:53.596590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:53.597257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.597579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.597858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.598648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:53.598956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:53.600480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd4e6e-276b-4124-aade-e8abb07f71dc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.614008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 344/694] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:53.619786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:53.734225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11250551-a954-442a-bcc5-02bba4c9529c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:53.771523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:53.772198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.772532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.772774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.773592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:53.774469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:53.775909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95263bc-e232-4375-8305-25e163c11556 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:53.784737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 351/695] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:53 2022] GET /identity/v3/role_assignments?user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:53.935077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb76b05-0c16-4202-98e4-f106b2b5e3d7 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.935077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb76b05-0c16-4202-98e4-f106b2b5e3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.935077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb76b05-0c16-4202-98e4-f106b2b5e3d7 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.936150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 345/696] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:53.942387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:53.942699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:53.942953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:53.963441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:53.973187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:53.977593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:54.050057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 352/697] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:54.070726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb76b05-0c16-4202-98e4-f106b2b5e3d7 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:54.187058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb76b05-0c16-4202-98e4-f106b2b5e3d7 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:54.189443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:54.224087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:54.224962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:54.225241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:54.225509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:54.237911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:54.246278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:54.247215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:54.248134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:54.250876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 346/698] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:54.260971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:54.371239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9bb02-f709-4e1c-bfb6-3a507bb084ad None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:54.411454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:54.412604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:54.413093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:54.413575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:54.414533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:54.415080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:54.416721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:54.432951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 353/699] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:54.438518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:54.585034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595950e-c816-4560-bbd9-e72d6b1d4b59 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:54.635427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:54.636143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:54.636555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:54.636828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:54.652596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:54.654059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:54.675672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:54.676658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:54.677577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:54.677577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:54.677577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:54.677577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:54.681299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 347/700] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:54.683374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:54.688214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:54.865390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22a96-2fd6-45d7-b5bc-72c396820c77 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:54.909955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:54.910553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:54.910829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:54.911098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:54.911813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:54.912114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:54.913508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb7a30-c0e4-4fd1-823d-417fe848e985 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:54.931179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 354/701] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:54.936687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:54.963375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.059421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a28a1d-94d4-461c-94cd-927e03b35c87 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.059758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a28a1d-94d4-461c-94cd-927e03b35c87 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.059967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a28a1d-94d4-461c-94cd-927e03b35c87 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.061348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 355/702] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:55.071951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.072503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.072991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.075112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6322a5d8-560d-4bb4-a90c-379d9c8b1e25 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.100673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:55.102809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:55.173194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.174083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.174197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.174451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.176944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.182982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 356/703] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:55.195716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.197238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.206406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:55.220617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:55.221605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:06:55.222620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:55.228824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 348/704] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:06:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:55.234611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.316959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc963908-64da-4e0c-89c6-9eaf6d1c47ba None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.351796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.352985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.353466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.353938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.355062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.356678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:55.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:55.372206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:06:55.373066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.375992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 357/705] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:55.381894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.430506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.440791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.487814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745fc0a-0d2b-43e1-a8f5-c92491606c1c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.506058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f235043-a33a-48cb-af0a-b7b127a106c5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.525140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.526129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.526637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.527134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.528042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:55.528566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.530141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90531484-97e5-483f-b06b-a37c74f93a89 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.535292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 358/706] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:55.542601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42712e-e990-4930-af7b-d08f3f812aea None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.542601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42712e-e990-4930-af7b-d08f3f812aea None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.542601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42712e-e990-4930-af7b-d08f3f812aea None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.544145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.544866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.545148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.545428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.546272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:55.546583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.546945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 359/707] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:55.547731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42712e-e990-4930-af7b-d08f3f812aea None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.548029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40da44dd-02d2-4985-96a5-469483f8b359 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.567620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 349/708] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:55.569051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.569276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.569554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.587607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:55.588880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:55.672396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 350/709] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:55.673278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.696644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42712e-e990-4930-af7b-d08f3f812aea None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.717951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.733713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.734893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.735171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.735442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.736613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:55.736933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.738716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.747222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 360/710] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:55.748173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:55.761359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:55.850398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e59f8b-fada-45fe-8f08-3e24f90f178c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.887743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d8916-5cc0-41c9-aa74-d09861858bd4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:55.933863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.934667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.934947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.935217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] PATH_INFO: `/v3/services/network` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.935874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:55.937270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:55.942262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:55.942918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:55.943194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:55.943465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:55.955574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:55.956663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:06:55.957491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:55.961761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 361/711] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:55.963229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.025755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.085955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28630d3-2c70-49ac-bb62-9dfef6f107be None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.124750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.125886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.125886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.126088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.126743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] RBAC: Adding query filter params (name=network) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:56.127050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.128510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.137896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 362/712] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:56.139207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.161733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.163507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.226914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.276334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 351/713] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:56.279971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.384984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2801e1-731a-4bc6-9969-d32d14ddcc90 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.387650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98d0ff-29d7-4d7b-9b9e-358aaba9edd0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.400285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.423766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.424740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.425026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.425299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.425877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.426749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.427291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.427848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.428784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] RBAC: Adding query filter params (type=network) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:56.429288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.431449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d1b7-ba5d-4c6b-8353-c536e4c01645 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.436525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.444334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.444911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 363/714] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/services?type=network => generated 124 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:56.446802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93f42-a55e-4660-a44a-004619b27ce9 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.447398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93f42-a55e-4660-a44a-004619b27ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.447899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df93f42-a55e-4660-a44a-004619b27ce9 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.449867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 364/715] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:56.455765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100191b8-b245-423e-a862-8864547be1d0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.456483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100191b8-b245-423e-a862-8864547be1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.456978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100191b8-b245-423e-a862-8864547be1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:56.457466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:56.458351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:56.459696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:56.459696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:56.459696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:56.459696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:56.459696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 352/716] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:56.465747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.489109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.497103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-100191b8-b245-423e-a862-8864547be1d0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:56.498674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-100191b8-b245-423e-a862-8864547be1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:56.595992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 365/717] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 15:06:56.623231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100191b8-b245-423e-a862-8864547be1d0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.667695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae8c47-0936-4762-aef4-c12cce317791 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.709220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.709894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.710177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.710448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.711176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:56.712049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.714194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d8bb5-62ec-42de-95db-f41990674125 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.729177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 353/718] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:56.730987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94997c-5129-4a29-9a96-664762df994a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.731343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94997c-5129-4a29-9a96-664762df994a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.731687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae94997c-5129-4a29-9a96-664762df994a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.733980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 354/719] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:56.734901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94997c-5129-4a29-9a96-664762df994a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.769918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100191b8-b245-423e-a862-8864547be1d0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.782783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.814567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.815831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.816424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.816901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.829204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:56.836125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8763498b-7178-4202-b94b-5af4b01195e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:56.839080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:56.840014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:56.848047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 366/720] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:56.849891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.850355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.850820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.865644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae94997c-5129-4a29-9a96-664762df994a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:56.872281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:56.873912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:56.909543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:56.910371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:56.910651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:56.910928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:56.919094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:56.922930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:56.942480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:56.943801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 355/721] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:56.945106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:56.953056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 367/722] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:56.954523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.063248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.073363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42528f01-3f1a-462b-b5e3-8299b5f9416e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.079936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc694186-bf3c-41f4-bafb-3ba8dcfbc0cf None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.117375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.118965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.119474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.120005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.120773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.121253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:57.121464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.121704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.121974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.122376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.122753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:57.123056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.124486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.124595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.127496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.134449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 356/723] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:57.135562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.141581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 368/724] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:57.142488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.260873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4b62d-046d-4b8c-9f6a-8968036fe5fe None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.264856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052dfcb-2e62-48f1-ba5a-5aeaf888fdde None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.298713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.299875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.300428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.301046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.306772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.307566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.307850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.308120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.316230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.318134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.321880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:06:57.330800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:57.331831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:57.332741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 357/725] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:57.333444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:57.340745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:57.341655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:57.342528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:57.342528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:57.342528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:57.342528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:57.344775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 369/726] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:57.345940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.359857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.412128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.460899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b722b8-15d2-4fa7-bb2b-5ac68308ec09 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.466715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2c056-9739-4ba3-850a-43de66af842f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.499690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.500680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.501163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.501654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.502598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:57.503099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.503450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.504096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.504414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.504690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.505510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:57.505815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.506038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.507239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.513173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 358/727] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:57.514014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.515823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 370/728] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:57.524406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.623174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2eb7c-79b8-480a-a3e9-51b256faf047 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.647591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.661344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.662019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.662299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.662570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.663298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:57.663653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.665191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.678393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 359/729] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:57.685032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:57.689626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.738964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47d4d5-5a7c-46ce-ad58-47873c542227 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.799334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.800133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.800443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.800714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.813110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.814568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.830767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be613c5-2fc8-4263-a4f2-732b80422992 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:57.845227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:57.846152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:57.847028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:57.847028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:57.847028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:57.847028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:57.847028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 371/730] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:57.853020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.872698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:57.875552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:57.875774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:57.876047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:57.891412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:57.895596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:57.925787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:57.926764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:57.927696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 360/731] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:06:57.933351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:57.978930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c298d-0325-4fc8-b0a7-fc322afbeb00 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.003608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.014988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.015652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.015933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.016324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.017582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:58.017905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.019501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.032902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 372/732] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:58.045844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.051885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.122265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0518d8-cad5-4d14-82e9-9b9e35a745f5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.159510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.160379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.160657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.160926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.162358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:58.162672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.164121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.184741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 361/733] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:06:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:06:58.194293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.306612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee504e-83f1-4082-9d82-87e886e226d1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.342769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.343611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.343891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.344164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.351815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.353944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:06:58.370349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:58.371414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:58.372375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 373/734] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:58.381555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.398424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.407383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a982a-030d-4674-895e-12fb2230d8ba None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.429268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.480767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.481810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.482338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.482753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.490973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34f7ae-bb3b-44dd-86b8-9d09ac8a5884 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.527995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.529275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.529275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.529686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.530628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:58.531036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.532744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec334f-c332-45d5-ac61-ecf8eb76ffdd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.552759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 374/735] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:58.554169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32765-6ed8-406d-8be5-ca9b4b94e68a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.554557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32765-6ed8-406d-8be5-ca9b4b94e68a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.554928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c32765-6ed8-406d-8be5-ca9b4b94e68a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.557235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 375/736] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:58.558081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c32765-6ed8-406d-8be5-ca9b4b94e68a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.561105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.562755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968dbdd6-aabd-4318-af07-9d75552ab69c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.665220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.669277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 362/737] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 15:06:58.670476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.670986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.671349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.688629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:58.689859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:58.693575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.699349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c32765-6ed8-406d-8be5-ca9b4b94e68a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.748600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.750021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.750482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.750903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.764584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 363/738] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:58.765766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.847361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.849135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.878337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25744ea1-ec19-49e0-a0de-3a83c7348dc9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:58.916323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.916323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:58.916994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:58.916994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:58.917418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:58.929127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:58.930778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:58.948464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:58.949565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:58.950590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:58.950590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:58.950590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:58.950590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:58.951725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 364/739] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:58.956332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:58.967915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 376/740] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 15:06:58.968489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:58.973028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.074811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4a6a4-d8dc-4355-aa79-62829770b1dc None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.091758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca7279-b86c-4167-9540-ce1c601535eb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.110866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.111640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.111918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.112186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.112928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.114400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.126491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.127207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.127495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.127755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.128517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:59.128823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.130165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70775a6b-53a0-42cc-b224-aad64adb4a23 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.136498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 365/741] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:06:58 2022] POST /identity/v3/services => generated 251 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:59.137813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.148916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 377/742] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:59.151300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39069bb1-d2b4-4049-a798-6b288b98c616 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.151849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39069bb1-d2b4-4049-a798-6b288b98c616 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.152345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39069bb1-d2b4-4049-a798-6b288b98c616 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.154322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 378/743] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:06:59.155323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39069bb1-d2b4-4049-a798-6b288b98c616 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.203317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.226294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.291347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39069bb1-d2b4-4049-a798-6b288b98c616 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.293257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed4074-43d8-40d4-b4ee-8963de8f7e60 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.336052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.337458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.338114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.338114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.345553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.346352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.346630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.346899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.347271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.350169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.361496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.363268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d761541c-ec61-466d-a0ab-9faaedd6bdf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:59.363664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:59.364596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:06:59.365534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 379/744] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:59.366541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.366809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.367080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.386444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:06:59.387211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:59.388390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:59.389502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:06:59.389502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:06:59.389502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:06:59.389502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:59.389785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:06:59.392434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 366/745] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:59.393415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.464285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 380/746] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 15:06:59.465277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.506902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.521894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36b9d-5ed4-42f5-8845-df640d237193 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.534881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.574112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.575307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.575745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.576527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.577275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:59.577589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.579050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.592151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 367/747] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:59.593156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.599393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd7997-40a1-4ab8-a43d-b24ffd00720c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.645714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.646775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.647688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.648174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.649417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:59.650108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.652382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.668127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 381/748] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:59.669469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.698209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc4386-b1ca-4eaf-89d7-8227d79a4989 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.732359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.733036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.733313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.733586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.734310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.735676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.751593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 368/749] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:06:59 2022] POST /identity/v3/services => generated 240 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:06:59.752240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.775117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.802609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c320d-2c78-4bff-a445-d347063a8571 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.819381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:06:59.874611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4331d-7196-4872-b375-eadde8ad51d6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:06:59.881040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.882051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.882560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.883043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.884138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=c1d39def9fcb44948c25c9139d2138b6) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:06:59.884712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.886364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.900209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 382/750] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/role_assignments?user.id=c1d39def9fcb44948c25c9139d2138b6&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:06:59.924346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:06:59.925179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.925458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.925727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.938480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:06:59.940012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-86e03dc3-ff3d-4d64-a8c0-528f652fe874 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:06:59.964917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:06:59.965864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:06:59.966750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 369/751] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:06:59.972244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:06:59.978774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07649a-612d-42fb-9606-27d88b4d751a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.979112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07649a-612d-42fb-9606-27d88b4d751a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.979640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07649a-612d-42fb-9606-27d88b4d751a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:06:59.981232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 370/752] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:06:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:06:59.987014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:06:59.987305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:06:59.987487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.020606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:00.021826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:00.114176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d966f-3454-40e0-bcaa-640070e8f58a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.114719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 371/753] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:06:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:00.115156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:00.141805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:00.160124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.160859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.161141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.161415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.162223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:00.162595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:00.164064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:00.178626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 383/754] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:06:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:00.185021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:00.223602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:00.278081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cffd1-07a9-4e17-bc50-ca68b40b1c7b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.329993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8da01-522d-44af-abb3-4890b13667d8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.339367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.340168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.340472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.340738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.351991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:00.360561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:00.361505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:00.362330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 372/755] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:00.369192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:00.408872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.409534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.409813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.410082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.410912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:00.411218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:00.412672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:00.423426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 384/756] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/role_assignments?user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:00.456671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:00.505037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f7607-2302-4a9a-9ab3-88759b18c177 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.541908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.542506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.542785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.543058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.543823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:00.544127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:00.545606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:00.555397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 373/757] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:00.560806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:00.693655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:00.701110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec6ff1-64d2-475c-a7d3-9103e370c33e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.743640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.744468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.744803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.745077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.766196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:00.767641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:00.791084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:00.792064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:00.793040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:00.793040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:00.793040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:00.793040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:00.793040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 385/758] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:00.797908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:00.917082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcaf5b-8632-4450-8fc4-8ef1b3ce9e35 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:00.941173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:00.961002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:00.961812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:00.962096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:00.962375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:00.963109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:00.963415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:00.964900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:00.980414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 374/759] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:00.985459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:01.148411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f73ce-0f46-4da2-83d2-5921aa13c49f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:01.197104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:01.198265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.198753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.199229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.213227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:01.214937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:01.232890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:01.233839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:01.234767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:01.237359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 386/760] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:07:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:01.243153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:01.382015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c32bcc-12b7-4dfd-bc30-f782b5df7244 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:01.429136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:01.429813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.430660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.430937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.431774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:01.432076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:01.433543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886ae1a-c10b-4bd7-b6fe-9f3d2244f29a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:01.455713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 375/761] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:07:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:01.461038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:01.478521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:01.623650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2341316-284e-48f0-875f-c8695ebaf9fd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:01.664119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:01.664892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.665170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.665441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.685296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:01.686816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:01.688643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6d3d-95ca-422f-81b5-79fcbe6ffb1d None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.689172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6d3d-95ca-422f-81b5-79fcbe6ffb1d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.689634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f6d3d-95ca-422f-81b5-79fcbe6ffb1d None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.696382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 376/762] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:01.697821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.698314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.698794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:01.705016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:01.705913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:01.706984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:01.706984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:01.706984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:01.706984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:01.706984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 387/763] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:01.711683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:01.737663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:01.738825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:01.790495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:01.816191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:01.855843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 377/764] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:01.881352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850dbf5-0a2e-4cfd-95c7-8a410a994c13 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:01.917740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:01.944883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:01.948164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:01.948471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:01.948744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:01.949485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:01.949790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:01.952435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:01.970793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 388/765] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:01.975964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.108896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b8854-da7a-4d7a-ae88-12429fe33486 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.155720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.156896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.157356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.157801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.158651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.161018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:02.182370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:02.183320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:02.184151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:02.187244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 378/766] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 15:07:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:02.189407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eac22-c9f6-4e43-a064-bf42de17e7a7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.197266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.209103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:02.241106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:02.243444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.244289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.244721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.245002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.255018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.256785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:02.286522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:02.287486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:02.288758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 389/767] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:02.293765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.322294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330dc6e-87d7-4b1f-9e30-de5409805f43 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.358359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.359262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.359714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.360143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.361863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:02.366293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.368614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.379268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 379/768] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:07:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:02.384651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.508629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab1f27-7998-413c-86ec-04d52c8ac8d3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.554478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.555187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.555465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.555735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.556552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:02.556862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.558295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.580130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 390/769] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:02.589782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:02.593037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.598972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032defa-5010-44e2-ae92-c8573a18d251 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.643642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.644545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.644920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.645290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.646104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:02.646508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.647338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:02.649322 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5198c1-9bed-4e11-956d-0bf330cb9ce2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.658686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 380/770] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:07:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:02.660842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbaa56-f970-4061-9d4c-f721cc09f55c None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.661197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbaa56-f970-4061-9d4c-f721cc09f55c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.661539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbaa56-f970-4061-9d4c-f721cc09f55c None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.663064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 381/771] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:02.663770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbaa56-f970-4061-9d4c-f721cc09f55c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.721765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6fec9-9ac9-4b86-bc33-1acfa9248320 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.758764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.759579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.759856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.760257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.762331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=aa1fb96699054e5c94fcc188fd9535be, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.763864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a82cb-cade-457c-bcae-c12314750f8f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.784729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 391/772] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:02 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:07:02.784965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbaa56-f970-4061-9d4c-f721cc09f55c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:02.785050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b38c-72a5-40fc-9ff5-42f193466b6d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.785050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b38c-72a5-40fc-9ff5-42f193466b6d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.785050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c937b38c-72a5-40fc-9ff5-42f193466b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.799351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c937b38c-72a5-40fc-9ff5-42f193466b6d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:02.801193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c937b38c-72a5-40fc-9ff5-42f193466b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:02.831008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:02.831806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.832088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.832725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.834077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] RBAC: Adding query filter params (interface=public, service_id=5597d2b5a5eb487995c85d8103711731, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:02.834675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:02.836985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d88d39f-ff77-4f6b-9e2e-0164dd87e7cf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:02.851466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 382/773] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:02 2022] GET /identity/v3/endpoints?service_id=5597d2b5a5eb487995c85d8103711731&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:02.854880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e58b-08b8-4e35-a2f6-02165f054dc0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.855189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e58b-08b8-4e35-a2f6-02165f054dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.855760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52e58b-08b8-4e35-a2f6-02165f054dc0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.857680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 383/774] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:02.858548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e58b-08b8-4e35-a2f6-02165f054dc0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:02.887630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 392/775] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:02.889076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:02.889519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:02.889802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:02.903255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:02.921945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:02.923134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:02.950130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.014897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 393/776] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:03.016273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.020491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52e58b-08b8-4e35-a2f6-02165f054dc0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.075442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.076131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.076440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.079659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.079659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.079659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.086074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 384/777] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:02 2022] GET /identity/v3/services => generated 772 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:03.088076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.203281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.224548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c5fc2-7c49-4f5a-98ff-67e5e5714308 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.234587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2426062-7588-478f-b8d0-71a040e977b3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.269838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.270897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.271319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.271734 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.292852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-796a55d6-983f-4e53-b558-27a3744451b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:03.301606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:03.302794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:03.303748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 385/778] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:03.304448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aefd6c-9a48-44f5-9635-6a6c4b7045aa None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.304579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aefd6c-9a48-44f5-9635-6a6c4b7045aa None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.304861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aefd6c-9a48-44f5-9635-6a6c4b7045aa None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.306399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.306660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 386/779] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:03.307494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.307987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.308433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.309241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.310788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.312260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.312908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.313340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:03.325517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:03.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:03.327889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 394/780] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/services/network => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:03.328763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.342076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:03.343859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:03.371636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.424926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 387/781] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:03.425700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.489734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9dcaae-9767-44d5-92a9-58268727e7e2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.532987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.533899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.534331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.534781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.535689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:03.536153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.537712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.537839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.547163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 395/782] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:03.548415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.568522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdd542-3e5c-4595-8d27-8700d5b47a4d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.617220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.618498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.618775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.619043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.620010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] RBAC: Adding query filter params (name=network) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:03.620787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.622442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.632375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 388/783] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:03.633637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.649657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.667952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2044-0877-4ccb-bf42-09ab1538496f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.703365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.704584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.705091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.705569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.716462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:03.724555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:03.725512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:03.726345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:03.734367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 396/784] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:03.735453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.754592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be449ebe-eccc-4543-97ea-0f48aab62949 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.790114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.791150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.791150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.791359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.799887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.802639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.804093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:03.822999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:03.823888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:03.824831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:03.824831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:03.824831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:03.824831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:03.824831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 389/785] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:03.825386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.843757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4ad3a-544a-4a8f-a7d9-6c11c650a3fc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.879582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.880647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.881141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.881664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.882655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] RBAC: Adding query filter params (type=network) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:03.883207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.884830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.890235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 397/786] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:03.891391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:03.920691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:03.935245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123520e-6f5d-4daa-ad37-06f09f9a07cb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:03.973318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:03.973985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:03.974262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:03.974528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:03.975274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:03.975573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:03.977667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:03.991274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 390/787] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:03.991274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.012671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02198d03-b0bc-4444-83f9-fb177f70e33d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.050887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.051975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.052594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.053117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.054096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:04.054608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.056265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.057611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.077120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 398/788] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:04.080655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.121808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df60d7d-456a-432c-9293-5ded609e402a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.155708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.167171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.167830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.168107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.168398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.169187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] RBAC: Adding query filter params (interface=public, service_id=cd890115291545ef9fb8566f15b158c0, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:04.169497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.171754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.180442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 391/789] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:03 2022] GET /identity/v3/endpoints?service_id=cd890115291545ef9fb8566f15b158c0&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:04.181481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.199672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e17e925-937d-49ff-bade-3bc046b1f8f3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.238208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.239339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.239826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.240436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.251937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.253773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:04.277161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:04.278146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:04.279066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:04.279066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:04.279066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:04.279066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:04.285795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 399/790] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:04.286939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.309953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f363e8b-c759-415e-b610-423465de4e21 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.334627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.355601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.357237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.357726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.358227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.368531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.371241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:04.391930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:04.392958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:04.393883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:04.397349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 392/791] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:04.399482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.411360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6e8565-8045-4887-8567-816ae083aaa8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.439251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.461021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.462778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.463604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.464313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.465548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.467765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.475525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 400/792] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:04.477422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.542356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd5f94-9385-49b2-8d60-be51f7b56c30 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.582922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.583859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.584288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.584851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.585734 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:04.586232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.587790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.607387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 393/793] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:04.609025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.685726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9589af-233d-40ec-bce9-7ab765ae8cc1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.690040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.731373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.732423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.732423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.732638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.733443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:04.733807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.735684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.757906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 401/794] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:04.764485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.768461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fb111-4996-4cc8-84f9-46f1101936ee None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.806173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:04.807100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:04.807380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:04.807654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:04.815468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:04.817022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:04.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:04.833107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:04.834358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 394/795] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:04.837934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:04.841941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:04.972410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3456f61-c010-413f-a2ae-90a80511c193 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.973901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f80d9e-d70e-4109-bbf6-1507dd4b1376 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:04.977406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.011529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.012205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.012531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.012860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.013736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:05.014051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.015494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.034789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 395/796] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:05.041640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.042688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.049657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.049965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.050240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.080378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.080378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-eb6b6a26-2487-45fb-99cc-531b5b960273 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:05.109713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:05.110653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:05.111541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:05.111541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:05.111541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:05.111541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:05.111541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 402/797] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:05.117529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f165e-e79a-4c77-81b4-623c0b18ffaf None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.117817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f165e-e79a-4c77-81b4-623c0b18ffaf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.118099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f165e-e79a-4c77-81b4-623c0b18ffaf None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.120258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 403/798] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:05.121002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f165e-e79a-4c77-81b4-623c0b18ffaf None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.180423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f1614-d2ae-42e8-995f-a1b03941bee8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.228963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.230174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.230531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.230949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.243910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.244678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.247381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.253112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f165e-e79a-4c77-81b4-623c0b18ffaf None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-96f2e4c5-fa33-4b4f-92bf-36b64dd29074 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:05.276135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:05.277283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:05.278391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:05.278391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:05.278391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:05.278391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:05.279537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 396/799] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:05.281166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.281648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.282024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.293691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.294625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.295082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.295526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.296477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:05.296978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.298603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.300772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:05.302169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:05.304365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.311821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 404/800] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:05.312910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.383868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 397/801] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:05.385484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.416842 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820fe4e-2548-4688-b858-00948bb5691b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.452486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.453399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.453844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.454122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.454847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:05.455153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.456563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.470200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 405/802] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:05.471079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.491224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.518271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4789b-df70-4262-aad4-4aafead94081 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.565329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.566398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.566828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.567276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.575865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.578016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.580659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:05.601812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:05.603347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:05.604206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:05.611354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 398/803] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:05.611354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.613543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f79d17-65e5-4ca1-8272-a58ef5d88d93 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.651619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.652363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.652651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.652928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.665165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:05.673726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:05.674667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:05.675500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 406/804] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:05.680455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.722507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce12e12-5e7d-4d61-9445-9dba16d483ce None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.761079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.764753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.765648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.765932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.766202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.773266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.775253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:05.790104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:05.790998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:05.791856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 399/805] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:05.792566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.793514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b581ba-16c8-4f8f-84d0-55798fecf949 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.833864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.834531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.834815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.835093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.835886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:05.836762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.838197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ad00a-563a-4d7f-819d-d1faee6a93f4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.850627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 407/806] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:05.852206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a20c-14f2-477f-ae91-192f49dfa879 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.852802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a20c-14f2-477f-ae91-192f49dfa879 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.853382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1a20c-14f2-477f-ae91-192f49dfa879 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.855123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 408/807] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:05.855729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a20c-14f2-477f-ae91-192f49dfa879 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.865411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:05.914118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e18dac-0ccf-47b7-8e64-17159cb331b9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:05.947465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:05.948126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:05.948434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:05.948708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:05.949452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:05.949758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:05.951131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:05.963333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 400/808] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:05.963333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:05.982822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1a20c-14f2-477f-ae91-192f49dfa879 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.018640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.019417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.019817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.020112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.020906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:06.021213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.022661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb8878-b878-4d3d-a76b-db02d1acc2ea None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.024521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.041220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 409/809] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:06.041220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.041220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.041220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.056825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:06.057990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:06.067165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c75d72-c47b-4031-a375-61e3bac4ac62 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.086830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.102699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.103589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.103866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.104136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/c1d39def9fcb44948c25c9139d2138b6/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.119246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=c1d39def9fcb44948c25c9139d2138b6, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.120729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.134257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 401/810] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:05 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/c1d39def9fcb44948c25c9139d2138b6/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:07:06.135062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.147293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 410/811] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:06.148622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.277210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683239e-dbec-424a-8720-f6aa990e9150 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.290814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52debb-1cc5-4990-89bf-83e3b60c863c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.316376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.317356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.317743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.318110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.325439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.331751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.333559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.342736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.344577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.345015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.345732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/ec47b745ff5a4ced9b47ed5ecd2b28cb/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.348025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=ec47b745ff5a4ced9b47ed5ecd2b28cb, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.349688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:06.354865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:06.355963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:06.357082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:06.357082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:06.357082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:06.357082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:06.358782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 402/812] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:06.359555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.377215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 411/813] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:06 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/ec47b745ff5a4ced9b47ed5ecd2b28cb/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:07:06.384653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.396683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.542392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d3259-52b2-4f3d-bb84-d996c5956aed None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.587166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.587997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.588277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.588578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.589240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.590656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:06.604731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:06.605726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:06.606553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 403/814] 10.222.0.45 () {56 vars in 1129 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:06.612292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.619055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71058d42-9ce7-4ff4-8461-e205c44c4407 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.675076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.675751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.676029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.676315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.677939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:06.678249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.679688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.693992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 412/815] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:06.701015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.752443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.773043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1089f4-d465-4c4f-963f-9ab8e9b293ea None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.790216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:06.811832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.812586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.812974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.813336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.814214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:06.814571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.816108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.822055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 404/816] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:06.829399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.886641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d953484b-dc55-41c3-a74b-2c05fdb94b94 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:06.922155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:06.922973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:06.923240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:06.923516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:06.931908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:06.933518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:06.964343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:06.965522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 413/817] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:06.966487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:06.974237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e5bee-f69a-4be4-b15c-94b9220d2b1a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.011791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.012663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.013030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.013370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.014141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:07.014511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.015962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5edda6e-7b89-494b-baa8-e99abcace9ee None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.021683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 405/818] 10.222.0.45 () {56 vars in 1124 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:07.023004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f544-46fd-4b0c-a352-c5a66ef61721 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.023835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f544-46fd-4b0c-a352-c5a66ef61721 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.023835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c4f544-46fd-4b0c-a352-c5a66ef61721 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.029204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 406/819] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:07.030080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4f544-46fd-4b0c-a352-c5a66ef61721 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.068353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.097480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872f3b4-49d4-4cec-8fff-bc244fc4a2c9 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.099982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.137393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.137846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.138202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.138551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.139480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:07.139854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.141464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7ba28-ee4b-42b9-8816-c52a2024b5b4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.161762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 414/820] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:07:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:07.163171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.163527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.163876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.172644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c4f544-46fd-4b0c-a352-c5a66ef61721 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.183387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:07.185061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:07.216353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.217535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.218384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.218835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.220327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.222867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.252719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 407/821] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:07.254717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.273361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 415/822] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:07.297400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.360536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.430267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.432233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755b367-dd62-4ef8-b3c7-8947f6c5fb65 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.456140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ee700-910e-4d14-b8e5-b6139675c536 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.501984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.503029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.503315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.503863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.514251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.515128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.515409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.515676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] PATH_INFO: `/v3/services/network` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.516368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.517801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:07.536927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:07.537975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 15:07:07.538802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:07.541819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 416/823] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:07 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:07.548217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.603644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.605516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.676621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9103-2670-4c18-b4f8-9938ca02d1dc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.681349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.718730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 408/824] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:07.720342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.720529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.720910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.721296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.722105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] RBAC: Adding query filter params (name=network) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:07.722511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.724057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.724824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.729714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 417/825] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:07.737249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.757016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:07.846281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf326c-1ebb-437d-90f6-4ba7da0a7f77 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.898113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801de78-f678-4a16-b489-8f9ddecaff12 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:07.903301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.903973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.904251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.904550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.905255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:07.905557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.913001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:07.921556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 418/826] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:07.927473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:07.955464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:07.956260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:07.957307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:07.957580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:07.977916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:07.979660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:08.014026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:08.016126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:08.018277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:08.018277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:08.018277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:08.018277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:08.018277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 409/827] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:08.024368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:08.074871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:08.088799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:08.125639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7805d9-39c0-4837-9a6c-b188663feaa0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:08.185092 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:08.186140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.186631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.187072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8110731-277b-43c5-a908-48c449f25079 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:08.187324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.188290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:08.189917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.222903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:08.223600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.223881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.224153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.225024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:08.225396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:08.227521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.240562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 410/828] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:08.242101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 419/829] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:08.248477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:08.393949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:08.419678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bdfc64-d994-41ba-b603-5d99a60c4449 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:08.482087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:08.482891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.483171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.483440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.490772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:08.492283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:08.506865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:08.507784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:08.508828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 411/830] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:08.515371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:08.628232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525dd8ff-48b9-484d-8832-26d28da3bdbb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:08.667235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:08.667924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.668205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.668505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.669306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:08.669618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:08.671077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44de4c-648e-4d58-9c53-7a01c4e36ba0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.674154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:08.688201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 420/831] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:08.696095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:08.817120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35518c1d-37ba-45c2-9fee-f3000cc1c573 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:08.857571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:08.858237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.858519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.858793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.859630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:08.859938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:08.861428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a243-48ab-467d-899b-0014ccb06edf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:08.868215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 412/832] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:08 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:08.891190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118a9d-06d8-4d4d-961a-bd2f0ac1c783 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.891190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118a9d-06d8-4d4d-961a-bd2f0ac1c783 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.891190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c118a9d-06d8-4d4d-961a-bd2f0ac1c783 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.892222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 421/833] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:08.898259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.899120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.899382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.904751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:08.918488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:08.919712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:08.966482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90cd03-8fd1-4159-845a-afc43573a0dc None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.966852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90cd03-8fd1-4159-845a-afc43573a0dc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.967037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90cd03-8fd1-4159-845a-afc43573a0dc None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.968827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 422/834] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:08.974828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:08.975148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:08.975384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:08.995266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:08.996626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:08.998841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 413/835] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:09.031654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.082800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 423/836] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:08 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:09.106953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.158767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d54d47-5a5f-410e-8d29-0dc25bac5ee2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.192523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.199190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.199993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.200275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.200589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.212243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:09.220276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:09.221221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:09.222062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 414/837] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:09.227729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.244650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad33d88-fd57-4d2b-a0cb-b3c18035d57d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.273084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.283683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.284712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.285136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.285550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.296579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:09.304477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:09.306133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:09.307150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:09.309485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 424/838] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:09.314748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.350704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ccee54-cd65-4157-a498-d56dffe7c560 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.395901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.396921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.397460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.398014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.399472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:09.399952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.401527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.415865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 415/839] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:09.417356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.429988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0781a64-923e-4d68-8a59-6df4755e0b50 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.465103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.465775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.466059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.466333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.467159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:09.467385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.468803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.479661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 425/840] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:09.484688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.486147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.544677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.551419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af2559-1ca6-490c-9f44-d6680b54ee05 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.590311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.591256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.591786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.592154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.605864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.607823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.613850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de34572-0c15-475a-af5a-88c85d238779 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:09.628849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:09.630017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:09.631212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:09.631212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:09.631212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:09.631212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:09.634341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 416/841] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:09.639912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.658233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.659061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.659339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.659609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.671201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.672871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:09.690944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:09.691890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:09.693245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:09.693245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:09.693245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:09.693245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:09.696448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 426/842] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:09.698626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.763041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108b72c-ef10-4bcd-9a5e-4938a53184e7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.774156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.810684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.811345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.811624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.811893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.812639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:09.812941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.814872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.826412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:09.831318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b22ec9-ea33-4dea-b3c4-024526690e42 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.835052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 417/843] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:09.837712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.874168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.874798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.875095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.875395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.876120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:09.876455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.877819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:09.894854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 427/844] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:09.895866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:09.946912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef50c0-57c5-477e-84d8-4f17ad59a0ea None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:09.985777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:09.987017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:09.987517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:09.987983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:09.996338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:09.998147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.002280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9a4dc7-7fbc-4c28-90b5-e694112bd2d1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:10.014657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:10.015601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:10.016555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:10.023493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 418/845] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:10.025001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.038186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.039069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.039316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.039587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.047871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.049672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.057830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:10.069618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:10.071163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 15:07:10.072089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:10.081375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 428/846] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:10.082303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.104631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.146128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a31c3-919a-4304-b4ab-052e393cc4b2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.185464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.186141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.186424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.186701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.187544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:10.187853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.188590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27807fa1-409b-48f9-bf6a-29df6fc82f9e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.189336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac60fd5-a2c8-4274-a4e4-7e031f1327e0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.209257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 419/847] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:10.210626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383f93-9c53-4ef8-9fba-ae476cb5fac0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.210938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383f93-9c53-4ef8-9fba-ae476cb5fac0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.211245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18383f93-9c53-4ef8-9fba-ae476cb5fac0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.216616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 420/848] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:10.216616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18383f93-9c53-4ef8-9fba-ae476cb5fac0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.231492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.232229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.232545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.232849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.233663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:10.233979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.235418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0c9ad-93e3-4bcf-bedf-443ffca920a9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.253853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 429/849] 10.222.0.45 () {56 vars in 1135 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:10.255110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.255398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.255672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.272540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:10.274257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:10.310690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.330364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18383f93-9c53-4ef8-9fba-ae476cb5fac0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.342574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 430/850] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:10.345440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.359468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.368495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.369152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.369430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.369776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.441974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.443358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.469716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a264f5d-1cee-4747-bd52-b69b5bb72c81 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.504456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.505100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.505385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.505658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.530064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.551679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 421/851] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:10.551679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.581995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.583443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.593768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.653884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705e29a-2b26-4a32-8659-14eb84422418 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.690579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.691462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 431/852] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:10.691576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.691663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.691905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.692640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.693973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.696645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:10.706351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:10.707330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:10.708163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:10.713883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 422/853] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:10.715042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.768600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.808501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c8330-1618-4e75-9895-0844625252d2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.818563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:10.829023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5bda1-0165-40d1-b839-dba4fb0dbec0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:10.850576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.851372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.852245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.852547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.863151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.864624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.865454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:10.866265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:10.866545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:10.866823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:10.877708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:10.879205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:10.883795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:10.884862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:10.885897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:10.885897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:10.885897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:10.885897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:10.887448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 432/854] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:10.888391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:10.902125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:10.903025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:10.903903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:10.903903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:10.903903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:10.903903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:10.903903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 423/855] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:10.904800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.005677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a2a92-93e3-4f88-add5-3f285463f0d7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.020099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.026010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9b9d-d999-4950-bd9a-87efd168ae58 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.063649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.064611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.065009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.065374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.066463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.066927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.069717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.084684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 433/856] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.085259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.085849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.086559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.086829 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.087107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.087830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.088131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.089549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.108232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 424/857] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.108232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.117007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.195154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeeedd-2a08-45d5-ab8f-f7ec6e09d912 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.215794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56838f49-6107-43b7-b54a-4489611800c3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.231042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.231707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.231986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.232256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.233007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.233310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.234677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.245941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 434/858] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.246733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.251079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.251742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.252017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.252333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.253036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.253336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.254700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d62d2-4d25-4323-b045-a546c5ae1e2e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.259813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 425/859] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.260954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e94-ad1c-4562-8716-b8f4a3c7e459 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.261237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e94-ad1c-4562-8716-b8f4a3c7e459 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.261508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c0e94-ad1c-4562-8716-b8f4a3c7e459 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.262791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 426/860] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:11.263561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c0e94-ad1c-4562-8716-b8f4a3c7e459 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.291642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.350239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.373374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544736c6-3b97-4525-9d7f-650942904d86 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.381676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121c0e94-ad1c-4562-8716-b8f4a3c7e459 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.419042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.419855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.420134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.420434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.420778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.421580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.421860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.422131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.428055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.429572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.429750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.431665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-f4e381d9-cd88-4a31-9776-e04cfeac76f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:11.445695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:11.446651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:11.447942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 435/861] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:11.448782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:11.452186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:11.453317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:11.453752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.453752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.453752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.453951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 427/862] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:11.453951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.469672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:11.470853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:11.542567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 436/863] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:11.543697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.548613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.566655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd7e528-4596-4195-85fc-961e691befe5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.603600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.604789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.605224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.605497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.606390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.606703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.608144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.622124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 428/864] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:11.623275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.636547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.665891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e648bd2-9eb7-4c21-b8bc-78851c9b645c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.700642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.701308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.701584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.701856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.702643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.702956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.704420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.716727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 437/865] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.717550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.733218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1c25cb-eee1-4bc8-9a03-6e7cfca8b928 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.769436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.770243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.770520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.770793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.771440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.772857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:11.786193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:11.787157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:11.787998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 429/866] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:11.790031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.803030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.830598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67134753-7a05-4e58-9c02-ca3affbb62b3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.865177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.865827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.866104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.866380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.867222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=c1d39def9fcb44948c25c9139d2138b6) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.867539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.868984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.870868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:11.876619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 438/867] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/role_assignments?user.id=c1d39def9fcb44948c25c9139d2138b6&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:11.878475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:11.922347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e609c55-a7f9-4d6b-9644-65f35a3f4c31 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:11.971877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:11.973150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:11.973622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:11.974097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:11.975484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:11.976032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:11.979571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:11.988342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 430/868] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/role_assignments?user.id=ec47b745ff5a4ced9b47ed5ecd2b28cb&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:12.023016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8031b892-5b67-447c-ab1f-fff6745520c7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:12.068756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:12.069507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:12.070126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:12.070126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:12.070825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:12.071134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:12.073278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c1556-684d-4033-a207-4c41ba7073ca None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:12.079106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 439/869] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:12.085238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:12.177729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:12.212785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c044f0c-c308-40ee-a92a-137f86e93526 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:12.249201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:12.250122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:12.250404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:12.250676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:12.251379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:12.251685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:12.253095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9972f-547f-4e22-8382-e4aae565f22c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:12.257899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 431/870] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:13.511394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07989-a066-4f12-a165-76626ad25237 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:13.511913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07989-a066-4f12-a165-76626ad25237 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:13.511913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b07989-a066-4f12-a165-76626ad25237 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:13.513656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 440/871] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:13.518852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4340a5-3a18-4960-8a8f-ec049f6e99fd None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:13.519264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4340a5-3a18-4960-8a8f-ec049f6e99fd None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:13.519429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4340a5-3a18-4960-8a8f-ec049f6e99fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:13.536830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bc4340a5-3a18-4960-8a8f-ec049f6e99fd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:13.538045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bc4340a5-3a18-4960-8a8f-ec049f6e99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:13.607092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 432/872] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:13.629435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b07989-a066-4f12-a165-76626ad25237 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:13.695120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:13.737138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b07989-a066-4f12-a165-76626ad25237 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:13.771371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:13.771963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:13.772247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:13.772550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:13.773802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:13.775187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c0673-87d5-45ae-a1fb-59bddb1d9513 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:13.794588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 441/873] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:13 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:14.255690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d408-b6ea-437f-96c3-e625ee9bfe97 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.256068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d408-b6ea-437f-96c3-e625ee9bfe97 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.256251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07d408-b6ea-437f-96c3-e625ee9bfe97 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.258420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 433/874] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:14.263638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.264171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.264692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.282116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:14.283606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:14.360946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82bd4-b633-40cd-872b-6b94bda49338 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.361559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82bd4-b633-40cd-872b-6b94bda49338 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.362030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f82bd4-b633-40cd-872b-6b94bda49338 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.363785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 434/875] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:14.366584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 442/876] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:14.369954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.370456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.370914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.391174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:14.392102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.392519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:14.455946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:14.461484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:14.472319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 435/877] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:14.491696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.508364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf27147c-5d80-4275-9f7a-7e689115a9cf None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:14.544210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:14.545054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.545333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.545605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.556869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5d664a01-fe1e-4cbe-a664-7aa4ca3f11de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:14.565068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:14.571441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 443/878] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:14.572407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4aefd-3121-49fe-8030-b35b90286b8e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.573153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4aefd-3121-49fe-8030-b35b90286b8e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.573153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4aefd-3121-49fe-8030-b35b90286b8e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.573153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 444/879] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:14.573153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4aefd-3121-49fe-8030-b35b90286b8e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.608036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de40fb4-d0fa-4d50-89b6-461564cbb26f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:14.646412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:14.647141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.647438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.647688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.648404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:14.649803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e01c47cd-ea24-4984-ab5e-9c79c7e77189 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:14.664422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:14.665423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:14.671582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 436/880] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:14.672404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.672404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.672404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.686314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:14.687494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:14.692404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4aefd-3121-49fe-8030-b35b90286b8e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:14.702337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:14.729027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:14.730037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.730387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.730724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.731527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:14.731894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:14.733367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:14.747192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 445/881] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:14.747192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.750448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:14.762919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 437/882] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:14.763670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.861742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cce098-e61d-4d6a-a859-e2782346448b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:14.865428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15ff14-9122-4797-95ca-2fe65e6c414b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:14.900663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:14.901699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.902196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.902534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.903247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:14.903555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:14.904952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:14.907027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:14.907815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:14.908096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:14.908395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:14.915266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 446/883] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:14.916257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.919427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:14.920929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:14.939256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:14.941196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 438/884] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:14.942238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:14.960334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:14.998206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:15.030338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39f8ef-47f9-4143-8d27-c3661a756255 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.054473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3864292-a13d-4bd8-a2e6-5148fb2da6d6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.068839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.069532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.070370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.070370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.071537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.073096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.090926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 447/885] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:14 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:15.091738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:15.109655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.110712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.111215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.111786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.112889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:15.113503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.115136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.121747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 439/886] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:15.220369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:15.229122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36835f8e-b489-4115-a0a5-29a4acaf746c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.270222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.270917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.271199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.271472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.272215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:15.272562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.274510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.294284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 448/887] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:15.297489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:15.406491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355583fc-9ec0-4eec-bcd6-9ee084dc4b61 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.408029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:15.444567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.445418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.445697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.446035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.453838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.455300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:15.470948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:15.471914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:15.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 440/888] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:07:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:15.478684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:15.585131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf544d3-c1e2-405e-b014-d039269354b8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.625609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.626283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.626562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.626835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.627669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:15.627974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.629437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.648204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 449/889] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:07:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:15.657413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:15.765551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25287019-42d1-4cba-aa55-a692b84ffff7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:15.802539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:15.803188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:15.803470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:15.803767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:15.834658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:15.881144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:15.883192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58fbee-384d-4f29-84be-7eee5100ccf4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:15.985792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 441/890] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:15 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:15.993543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.009748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.055462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4b39-4eb5-48b4-9b8d-92b6434a22c0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.055658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4b39-4eb5-48b4-9b8d-92b6434a22c0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.055936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b4b39-4eb5-48b4-9b8d-92b6434a22c0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.061154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 442/891] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:16.062447 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.062730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.063003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.085308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:16.086548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:16.116164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476e663-4682-4dd0-adb2-6cae3ed0782d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.156629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.158363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.159564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.159906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.160244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.176335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 443/892] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:16.181142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.182778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.194399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:16.202602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:16.203683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:16.204788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:16.204788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:16.204788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:16.204788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:16.206992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 450/893] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:16.212033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.291323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.320021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e0073-061d-4a7a-a3ac-867a04d385bd None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.333728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a39719-3e06-40c5-92f9-f2523b2d33eb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.360435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.361232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.361513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.361789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.362557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.364347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.375760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.376784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.377295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:16.377534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:16.378573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:16.379430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:16.381015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.381962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.382486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.383862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 444/894] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:16.384486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.384741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.402320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 451/895] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:16.403545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.444766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.510063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee345da-628a-40bb-9ee3-6f8da999560c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.511953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313f6-d295-445c-9a3c-a511e664a49e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.537303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.549711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.550561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.550780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.551056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.557789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.558976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.559105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.559583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.560211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.560706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.561410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.561931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.564107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.569686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 445/896] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:16.575431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:16.580205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:16.581117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:16.582005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:16.586165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 452/897] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:16.587000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.693250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0a422-6c37-4809-8609-719abbaf41de None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.714895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edcc0f-c5e3-4048-be34-136aeb8c7353 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.729578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.730244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.730706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.730825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.731561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.731877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.733351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.743401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.750820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 453/898] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:16.751878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.766075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.767101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.767385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.767659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.768395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.768965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.770927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.778451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 446/899] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:16.784195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:16.810772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:16.881904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c2e7-1ee4-4450-b77f-5096eaf9177a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.914756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fd057-dd2b-40f5-acde-86607c43f1f9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:16.926242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.927221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.927714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.928203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.929273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] RBAC: Adding query filter params (role.id=a0214636f75e478d847025a8f8f9865d, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=c1d39def9fcb44948c25c9139d2138b6) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.929840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.931473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.943474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 454/900] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/role_assignments?user.id=c1d39def9fcb44948c25c9139d2138b6&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=a0214636f75e478d847025a8f8f9865d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:16.953542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:16.954216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:16.954494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:16.954766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:16.955566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] RBAC: Adding query filter params (interface=public, service_id=249d4a31224e411ebebd82fe68e204d2, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:16.955878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:16.957341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b65170-38a6-47fd-aee4-a841a48aceb3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:16.964880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 447/901] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/endpoints?service_id=249d4a31224e411ebebd82fe68e204d2&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:16.970506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:17.096977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8da4-8521-4d81-959d-b4e242d41c37 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:17.108224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:17.135925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:17.136556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.136838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.137111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.137858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:17.139249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96d16e-eab0-40d0-a5fd-58e5a034b93c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:17.146042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 455/902] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:16 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:17.466060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562602-b260-4f80-b064-8362685d9a10 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.466427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562602-b260-4f80-b064-8362685d9a10 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.466616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562602-b260-4f80-b064-8362685d9a10 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.468182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 448/903] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:17.472952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ddcf8-6905-4692-aaa7-a804957a2554 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.473242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ddcf8-6905-4692-aaa7-a804957a2554 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.473408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.473570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ddcf8-6905-4692-aaa7-a804957a2554 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.473725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.473924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.475111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 456/904] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:17.480511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.480799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.481080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.492960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:17.494205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:17.498641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:17.500032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:17.541765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:17.576430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 449/905] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:17.580870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 457/906] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:17.594849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:17.606458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:17.711552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932fc12-0f10-4901-8466-bd7b311f9f49 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:17.745445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386fd9e-11ab-4c37-bf1d-5aec4d90c595 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:17.747549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:17.750450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:17.754516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.754516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.754516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.754516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:17.754516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:17.768971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:17.769968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:17.771062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 450/907] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 15:07:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:17.778667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:17.785732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:17.786509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.786869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.787251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.788057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:17.790098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:17.812373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 458/908] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:17 2022] POST /identity/v3/services => generated 241 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:17.841591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:17.910809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718fe471-029a-4434-84e9-628b32b42a4c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:17.957417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:17.958507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:17.959032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:17.959585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:17.960538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:17.961182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:17.962996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:17.969419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 451/909] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:17.976937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:18.088822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cee5bb-cae4-4c1b-99f0-fd5d84b74b0a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:18.127499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:18.128168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:18.128472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:18.128747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:18.129450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:18.129755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:18.131123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:18.140666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 459/910] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:18.142478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:18.249185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055a44d-515a-4814-bc29-f157a63e5099 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:18.251124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:18.287446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:18.288526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:18.289571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:18.290060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:18.291052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] RBAC: Adding query filter params (interface=public, service_id=7f31f2f858e343e2aeae8649d7c2dd25, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:18.291636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:18.293289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb3401-f1a6-4b21-acbf-f3a1574b623a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:18.299222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 452/911] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:18 2022] GET /identity/v3/endpoints?service_id=7f31f2f858e343e2aeae8649d7c2dd25&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:18.304628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:18.335552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:18.413583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42060ae-e9d0-430b-aecc-200d4e6126b2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:18.448885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:18.449895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:18.450384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:18.450864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:18.451811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:18.453488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afd8ae-a1d8-4cc5-b35f-1805700494be None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:18.460130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 460/912] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:18 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:19.258266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fe0b-427f-4800-a9ac-1f37a7460e58 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.259277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fe0b-427f-4800-a9ac-1f37a7460e58 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.259799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df37fe0b-427f-4800-a9ac-1f37a7460e58 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.261648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 453/913] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:19.267357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43f96-ef17-4d20-affd-727c05a5d683 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.268168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43f96-ef17-4d20-affd-727c05a5d683 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.268683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de43f96-ef17-4d20-affd-727c05a5d683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.286650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9de43f96-ef17-4d20-affd-727c05a5d683 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:19.288183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9de43f96-ef17-4d20-affd-727c05a5d683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:19.358235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 461/914] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:19.381621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37fe0b-427f-4800-a9ac-1f37a7460e58 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.420641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d315dc-44c2-4e10-a1dd-94c666c1d5b0 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.420945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d315dc-44c2-4e10-a1dd-94c666c1d5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.421210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d315dc-44c2-4e10-a1dd-94c666c1d5b0 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.422905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 462/915] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:19.427635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.428242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.428242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.444703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:19.446305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:19.460735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:19.508285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df37fe0b-427f-4800-a9ac-1f37a7460e58 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:19.537479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 463/916] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:19 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:19.546198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:19.547070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.547354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.547679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.559207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:19.560546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:19.568705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:19.569715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:19.571321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 454/917] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:19.577708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.609068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:19.689671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0138f-9964-4d20-89e5-c445c2607bda None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:19.703703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e568eb-ff05-45da-9eae-c9b8f82249f3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:19.726849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dded33-0100-457e-975a-531b203e340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:19.727460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dded33-0100-457e-975a-531b203e340e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.727744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dded33-0100-457e-975a-531b203e340e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.728015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dded33-0100-457e-975a-531b203e340e None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.728806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dded33-0100-457e-975a-531b203e340e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:19.729111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dded33-0100-457e-975a-531b203e340e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:19.730755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dded33-0100-457e-975a-531b203e340e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:19.744102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 455/918] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:19.745192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dded33-0100-457e-975a-531b203e340e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.754203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:19.755008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.755287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.755559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.756221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:19.758591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:19.770815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:19.772043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 15:07:19.773138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:19.779071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 464/919] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:19.780243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.816861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:19.864975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:19.880513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dded33-0100-457e-975a-531b203e340e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:19.898697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b6d32-35d1-4d4b-bf90-ad449d9883a5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:19.918031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:19.918918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.919176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.919451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.931965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:19.933535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:19.949140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:19.950207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:19.950694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:19.951195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:19.952122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:19.952328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:19.953193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:19.954061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:19.954061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:19.954061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:19.954061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:19.955043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:19.956657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:19.958286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 456/920] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:19.959379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:19.965167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 465/921] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:19.973292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.055572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.094963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e86b44-6d0f-4ec8-89b3-751b7d928db3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.098668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4fe73-c305-4010-bf0c-ae0913ed697d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.133455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.134049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.134330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.134625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.135314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:20.135630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.137031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dbcbd-b73a-42b3-bfe8-78dedd812755 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.139052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.139818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.140105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.140410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.141131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:20.141439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.144488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 466/922] 10.222.0.45 () {56 vars in 1114 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:20.144652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.144761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53a832-9983-4ecb-821b-9ad3e86198be None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.144761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53a832-9983-4ecb-821b-9ad3e86198be None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.144761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53a832-9983-4ecb-821b-9ad3e86198be None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.145606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 467/923] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:20.151427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.151716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.152435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.154427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.162764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 457/924] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:20.163620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.169258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:20.170440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:20.248333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 468/925] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:20.248333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.274317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41844d-f7e9-452d-9d1b-57d4413b8e37 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.309574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.310235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.310587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.310871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.311610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.313026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.331860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.344327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 458/926] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:20.344327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.373054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d962c-a5f1-4ed9-86df-0823ec3e2d83 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.414851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.438489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.439318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.439655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.439934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.452039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.453888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.472545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e067a-dcc2-404c-bb07-64d934aca457 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:20.476153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:20.477254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:20.478203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:20.483187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 469/927] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:20.484140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.518313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.519361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.519746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.520127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.520967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.522637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:20.536776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:20.539074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:20.541142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 459/928] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:20.549754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.615875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c598f84-3e2f-483a-8abb-daead0c4fe6b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.648290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.663228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.664374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.664699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.665035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.666644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:20.666963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.669117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.681378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383862f-7118-4bb9-8fb4-ceb3ceea127b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.693246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 470/929] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:20.698480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.712798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.723912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.724537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.724815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.725085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.725779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:20.726092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.727498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04750b-30b1-4372-b10f-9017e0931b3e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.732901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 460/930] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:20.734149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2f9f-df0e-4329-87f6-c1cd55d6de90 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.734434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2f9f-df0e-4329-87f6-c1cd55d6de90 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.734704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2f9f-df0e-4329-87f6-c1cd55d6de90 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.736035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 461/931] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:20.740954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2f9f-df0e-4329-87f6-c1cd55d6de90 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.809906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa8cf-dfe7-4ede-a90e-24ecf981cc24 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.848326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.849232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.849518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.849785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.857393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2f9f-df0e-4329-87f6-c1cd55d6de90 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:20.861384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.862820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1e627f77-6a22-4a6e-a2fc-c7e43ea7d52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:20.881452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:20.882600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:20.883490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:20.883490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:20.883490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:20.883490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:20.883490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 471/932] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:20.884087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.884186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.884481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.897120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:20.897781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:20.898058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:20.898329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:20.899034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:20.899336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:20.900827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:20.901539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:20.902988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:20.910417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 462/933] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:20.910417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:20.940028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.958549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:20.982459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 472/934] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:20.983675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.024314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac74a97-a34a-46c1-9cd1-7f78474a73a7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.059658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.062253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.062546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.062822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.063548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:21.063872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.067991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.084274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 463/935] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:21.084274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.107825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc2ef-6f52-4722-af21-3af09d9b8c05 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.145303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.146058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.146416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.146738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.147596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] RBAC: Adding query filter params (interface=public, service_id=b28640ede35f4038b0ddb1940c65b41c, region_id=RegionOne) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:21.147909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.149359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.157940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 473/936] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:20 2022] GET /identity/v3/endpoints?service_id=b28640ede35f4038b0ddb1940c65b41c&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:21.159179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.192812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.198991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42087f87-7587-48a3-a4c4-1e468def6c72 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.208560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.239201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.240100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.240410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.240684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.241332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.242747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:21.255112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:21.256041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:21.257881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 464/937] 10.222.0.45 () {56 vars in 1132 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:21.258739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.274603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e00af-58ac-4245-ba1b-f33400bd2f8e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.311366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.312214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.312524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.312801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.319586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.321113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:21.336703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:21.338025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 474/938] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:21.339179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.366612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325ced-9fdb-41d9-8593-4d0d521e130d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.400902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.401672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.402161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.402351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.403002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.404532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.420385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 465/939] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:21.420385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.455533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.463084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.481308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573662f-b34a-42e0-85b6-134681837a1d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.533543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.534236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.534521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.534798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.535037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f1789-8e9e-4361-b8e7-5fee82b94286 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.535523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:21.535833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.537572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.545681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 475/940] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:21.551763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.588243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.589736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.590190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.590542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.591683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:21.592506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.594496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.611788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 466/941] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:21.617512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.673015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0c21e-b234-499c-a980-72a71662c787 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.716741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.717991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.718347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.718696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.719765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:21.720145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.721902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.729515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 476/942] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:21.734931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:21.737230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.746471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71935b-536d-43a4-a38b-b5c92e349a6c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.768644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:21.792992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.794186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.794647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.794990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/c1d39def9fcb44948c25c9139d2138b6/roles/a0214636f75e478d847025a8f8f9865d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.798043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=c1d39def9fcb44948c25c9139d2138b6, role_id=a0214636f75e478d847025a8f8f9865d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.800152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7726e0-e300-4595-aab0-19368cd5deb4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.818800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 467/943] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:21 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/c1d39def9fcb44948c25c9139d2138b6/roles/a0214636f75e478d847025a8f8f9865d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:07:21.868394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31cc5-43fa-4994-9fca-13b15fdbd2b1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:21.911294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:21.912229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:21.912723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:21.913187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:21.914135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:21.915684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc08ed-c15d-4f2b-9685-d615e7670dc8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:21.942394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 477/944] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:22.700040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be324e7-7df2-4388-be29-9abf7afe776b None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:22.700589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be324e7-7df2-4388-be29-9abf7afe776b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:22.700670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be324e7-7df2-4388-be29-9abf7afe776b None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:22.702221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 468/945] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:22.707758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:22.708197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:22.708360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:22.725464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:22.726682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:22.805087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 478/946] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:22.829000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be324e7-7df2-4388-be29-9abf7afe776b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:22.935907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be324e7-7df2-4388-be29-9abf7afe776b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:22.971632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:22.972570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:22.973059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:22.973514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:22.974394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:22.976027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:22.977611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING keystone.server.flask.common [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 15:07:22.998199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.sql.core [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:07:22.998199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 15:07:22.998199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [parameters: {'id': 'e674238f1440438b8b90dccda69f6f49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 15:07:22.998199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80365) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e674238f1440438b8b90dccda69f6f49.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e674238f1440438b8b90dccda69f6f49. May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application conn.query(q) May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:07:23.007199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result.read() May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 24 15:07:23.008642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return project_ref.to_dict() May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application next(self.gen) May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application yield resource May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application next(self.gen) May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise self.value May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application session.commit() May 24 15:07:23.009833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.session.flush() May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application compat.raise_( May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:07:23.010903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:07:23.011800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application conn.query(q) May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application result.read() May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:07:23.012746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [parameters: {'id': 'e674238f1440438b8b90dccda69f6f49', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:23.013800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:23.014795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.Conflict( May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e674238f1440438b8b90dccda69f6f49. May 24 15:07:23.015689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:23.018035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 469/947] 10.222.0.45 () {60 vars in 1140 bytes} [Tue May 24 15:07:22 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:23.024564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:23.026715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:23.146603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed9c014-15e8-416f-a85e-fe7a7bd1e49f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:23.182897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:23.183938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:23.184426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:23.184861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:23.192136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:23.193767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-fe8b952e-5041-47e5-a7e1-a75e6f748241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:23.207983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:23.209115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:23.210195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:23.212362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 479/948] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:23.221730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:23.240278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:23.332585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2014f-f2f8-49f4-9c11-fd5661267a31 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:23.367890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:23.368584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:23.368868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:23.369142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:23.369900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:23.370208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:23.371862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c605bb30-2493-4695-a8f0-048f7efc237d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:23.392239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 470/949] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:23.760754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03557c3-b5ba-4b22-978b-6af20847ab67 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:23.761273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03557c3-b5ba-4b22-978b-6af20847ab67 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:23.761273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03557c3-b5ba-4b22-978b-6af20847ab67 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:23.762906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 480/950] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:23.768381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:23.768812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:23.768959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:23.787519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:23.789895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:23.862304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 471/951] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:23.884928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03557c3-b5ba-4b22-978b-6af20847ab67 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:23.948658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:23.996958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03557c3-b5ba-4b22-978b-6af20847ab67 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.037015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.037742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.038021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.038294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.039046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.040503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1989456b-29c7-495d-9a2a-07e7e131d2bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:24.052900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:24.054077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:24.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 481/952] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:24.060232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.096681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.169363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adb221-1f2a-4f60-9cbb-291e8b6b4fd7 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.169690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adb221-1f2a-4f60-9cbb-291e8b6b4fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.169923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31adb221-1f2a-4f60-9cbb-291e8b6b4fd7 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.174946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 482/953] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:24.176821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.177290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.177762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.184218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57964b9-a693-47f1-914d-29325d5c8d2c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.195750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:24.197257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:24.220869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.221567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.221865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.222143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.222852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:24.223161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.224587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.234923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 472/954] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:24.236598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.284320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 483/955] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:24.299413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.321373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.360624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.366227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4583346-cec9-440b-b2de-33eae5a910df None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.401262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.401923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.402207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.402485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.403189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:24.403499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.404923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20919c48-8df6-40b4-b785-5c73e047806c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.409782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93918-8c83-4fa1-a595-70d9bc425424 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.415046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 473/956] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:24.415046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e57d8-89b1-4b73-9ef5-64b300145547 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.415046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e57d8-89b1-4b73-9ef5-64b300145547 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.415046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e57d8-89b1-4b73-9ef5-64b300145547 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.415046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 474/957] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:24.418089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e57d8-89b1-4b73-9ef5-64b300145547 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.449828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.450649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.451106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.451240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.462112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-72b2bc7e-72a0-4f6a-807a-dd9454cad2a2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:24.470278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:24.471234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:24.472071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 484/958] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:24.473461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.473461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.473461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.490276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:24.491463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:24.533709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e57d8-89b1-4b73-9ef5-64b300145547 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.561087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 485/959] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:24.561981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.578205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.590206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.590868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.591151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.591415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.592157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.593559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.613942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 475/960] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:24.616511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.617775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.702988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f791020-cd51-4f18-b50f-6a2d66b5a105 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.757590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eb168-a123-4ce6-a9db-45d61823d590 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.759099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.760400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.762018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.762990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.763987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:24.764567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.766897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.779396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 486/961] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:24.786225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.802291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.803094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.803380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.803651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.805594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.807767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:24.821056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:24.822079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:24.823441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 476/962] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:24.827986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:24.891060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.916396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9be1a-bde8-40d4-a9ec-a08313a39e6e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.920676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:24.953347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:24.954135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:24.954416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:24.954689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:24.965186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a1f93-2943-4dc9-bdbd-9f62f2d11221 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:24.967769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:24.969521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:24.992551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:24.993692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:24.994793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:24.994793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:24.994793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:24.994793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:24.997379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 487/963] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:25.002834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.012698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.013495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.013839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.014199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.015029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:25.015398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.017133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.022484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 477/964] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:25.028173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.121155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88364c4-8478-4f4c-a7fa-2e30b3401c21 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:25.157966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.158635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.158917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.159194 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.159915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:25.160222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.161878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.174377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812d8d0c-f766-459a-b336-7e14e9b454f4 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:25.175245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 488/965] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:25.180542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.189206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:25.202549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:25.221455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.222122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.222401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.222673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.223388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:25.223694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.225105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.236386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 478/966] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:25.307443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89494d40-8ac5-4138-ae47-0f484d79e9ca None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:25.348048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.349268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.349758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.350238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.359311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.361277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:25.378290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:25.379302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 15:07:25.380235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:25.383640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 489/967] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:25.392387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.500607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d5410-df12-4a62-ad67-26ac592b69fa None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:25.541003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.541919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.542344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.542904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.543918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:25.544404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.545998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77b260-4c57-4a49-a7f2-4ac4c7dbe8a6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.568673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 479/968] 10.222.0.45 () {56 vars in 1149 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:25.571148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.656795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:25.681937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7ae79-7644-4fd0-989c-36ce36a00c22 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:25.717717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:25.718606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.719054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.719482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.783738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60784-f08f-4944-b838-4c58461ad677 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.784027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60784-f08f-4944-b838-4c58461ad677 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.784323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab60784-f08f-4944-b838-4c58461ad677 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.785929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 480/969] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:25.792021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:25.792327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:25.792717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:25.793642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:25.795938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:25.810708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:25.812173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:25.847761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:25.884655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:25.912575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 481/970] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:25.922593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 490/971] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:25.931926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:25.937703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.039929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270b97f-0fde-4d37-ad0a-b94ad155501b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.081886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a3cb1-1465-4121-a86f-de82784d3e88 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.089008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.090101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.090590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.091065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.102606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.104813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.123078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.123757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.124032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.124327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:26.124713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:26.125564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:26.126467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:26.126467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:26.126467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:26.126467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:26.126685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.126685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.127989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING keystone.server.flask.common [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 15:07:26.129198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 482/972] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:26.138399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.148870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.sql.core [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:07:26.148870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 15:07:26.148870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [parameters: {'user_id': '6c4f5fdd677a4525813f5f88971de6e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 15:07:26.148870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80366) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 15:07:26.157539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application conn.query(q) May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application result.read() May 24 15:07:26.175017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:07:26.175934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application next(self.gen) May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application yield resource May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application next(self.gen) May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self.session.rollback() May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self.force_reraise() May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise self.value May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application session.commit() May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 15:07:26.176907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._prepare_impl() May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self.session.flush() May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._flush(objects) May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application compat.raise_( May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application flush_context.execute() May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rec.execute(self) May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 15:07:26.177811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application result = connection._execute_20( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 15:07:26.178808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application result = self._query(query) May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application conn.query(q) May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application result.read() May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 15:07:26.179665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c4f5fdd677a4525813f5f88971de6e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:26.180525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 24 15:07:26.182576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 491/973] 10.222.0.45 () {60 vars in 1131 bytes} [Tue May 24 15:07:25 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:26.184590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.233574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:26.279940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf162f2-bbee-4c65-87c2-3ef303ebe3b7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.301426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1b614-2786-4f0d-ab34-dc14a8c3ebbd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.325313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.326062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.326682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.327030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.328238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:26.328645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.330658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.340350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.341497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.341974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.342427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.349251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 483/974] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:26.350899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.356866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.358770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:26.375204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:26.376178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 15:07:26.377171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:26.380191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 492/975] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:26.386567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.448750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:26.472894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b827a9-3d17-4a1e-af48-c1de7e6c4de5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.511244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:26.515954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151a61b-23fd-4ca5-a0be-334823ef0d54 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.519019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.520701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.520987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.521259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.530513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.531996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:26.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:26.552277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:26.553516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 484/976] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:26.559413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.560659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.561170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.561715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.562154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.563123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:26.563579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.565206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.620611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 493/977] 10.222.0.45 () {56 vars in 1101 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:26.682436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09747ad8-a806-4a00-baa6-cf59f43d2aa7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.721569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.722408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.722787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.723157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.724048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:26.724504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.726703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3969dd-6193-41e7-8d8c-2324b5ad9bb5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.728766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:26.742955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 485/978] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:26.747994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:26.868268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cb121-bd3b-402e-9fc6-176e9b53de46 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:26.904745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:26.905654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:26.906082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:26.906501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:26.907497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] RBAC: Adding query filter params (role.id=a0214636f75e478d847025a8f8f9865d, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=c1d39def9fcb44948c25c9139d2138b6) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:26.907958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:26.909557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df838788-7e1f-4dfb-9c38-664d0847be7d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:26.916850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 494/979] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:26 2022] GET /identity/v3/role_assignments?user.id=c1d39def9fcb44948c25c9139d2138b6&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=a0214636f75e478d847025a8f8f9865d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:27.033823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8818b-dade-4d4a-841c-aceb44bde139 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.034346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8818b-dade-4d4a-841c-aceb44bde139 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.034346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8818b-dade-4d4a-841c-aceb44bde139 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.035872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 486/980] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:27.042019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.042335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.042586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.061179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:27.062737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:27.084905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:27.144234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 495/981] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:27.167910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8818b-dade-4d4a-841c-aceb44bde139 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.276024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8818b-dade-4d4a-841c-aceb44bde139 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.311697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.312469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.312751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.313022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.313700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.315136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:27.328421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:27.329499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:27.330391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:27.332389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 487/982] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:27.336781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.439826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a537-3cd6-40d2-9cea-6f197713b9ed None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.474591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.475267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.475548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.475824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.477127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:27.477439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.478822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30451e5b-2720-401b-b9cb-dbf9a5b58853 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.488580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 496/983] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:27.490140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.497412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11475-5822-47a3-804a-3bdbe2cb7424 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.497700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11475-5822-47a3-804a-3bdbe2cb7424 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.497977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11475-5822-47a3-804a-3bdbe2cb7424 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.499445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 497/984] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:27.504369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.504668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.504938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.521327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:27.522561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:27.533382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:27.556356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:27.601376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc060665-1468-42de-a6a8-184241a83a25 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.604858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 498/985] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:27.626379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.637972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.638950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.639228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.639499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.640200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:27.640534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.641910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.646792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 488/986] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:27.653294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.731389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace476c-bf2e-4137-9c8a-37258900c674 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.762270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeeb6ff-3ed1-4d23-8782-b5de6536d762 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.767224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.767900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.768185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.768494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.769254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.770667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.786786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 499/987] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:27 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:27.800885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:27.823704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.824680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.824964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.825240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.826061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] RBAC: Adding query filter params (interface=admin, service_id=b28640ede35f4038b0ddb1940c65b41c, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:27.826376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.827809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a6140-be6a-45c6-b138-cc56284647ab None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.839114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 489/988] 10.222.0.45 () {56 vars in 1249 bytes} [Tue May 24 15:07:27 2022] GET /identity/v3/endpoints?service_id=b28640ede35f4038b0ddb1940c65b41c&interface=admin®ion_id=RegionOne => generated 193 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:27.841440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:27.845651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:27.952351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23bf631-c262-4d04-92e9-4fb8e599ac44 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:27.988825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:27.989495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:27.989771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:27.990050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:27.990727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:27.992668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f171b898-1eab-41ef-91ee-6cd9d295422c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:27.999229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 500/989] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:27 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:29.183936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfca8b-79a6-4b3a-940e-c03824b4046f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:29.184518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfca8b-79a6-4b3a-940e-c03824b4046f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:29.184518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfca8b-79a6-4b3a-940e-c03824b4046f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:29.185919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 490/990] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:29.191002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:29.191387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:29.191564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:29.208412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:29.209585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:29.271135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:29.277874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 501/991] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:29.299964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfca8b-79a6-4b3a-940e-c03824b4046f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:29.400554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfca8b-79a6-4b3a-940e-c03824b4046f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:29.433971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:29.434745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:29.435023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:29.435296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] PATH_INFO: `/v3/services/image` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:29.435943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:29.437351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:29.448811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:29.449834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:29.450687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:29.452206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 491/992] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 15:07:29 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:29.455990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:29.557724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14422b0d-3027-4534-9ce4-b6f8e78dd534 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:29.591480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:29.592152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:29.592454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:29.592727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:29.593428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:29.593731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:29.595107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1878522-94c5-4481-b754-2f2dba081ed3 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:29.605236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 502/993] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:29.606695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:29.655342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:29.710063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dfdb7-f2d5-4963-b9f1-6cacf97b848d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:29.744883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:29.745546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:29.745827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:29.746098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:29.746801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] RBAC: Adding query filter params (type=image) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:29.747107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:29.748510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782bc12-d1ec-4e12-9cfd-2564fb4453ff None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:29.759921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 492/994] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:30.027285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac198f4-61ab-4b62-ac26-6881b13b925e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.027678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac198f4-61ab-4b62-ac26-6881b13b925e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.027885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac198f4-61ab-4b62-ac26-6881b13b925e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.029384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 503/995] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:30.035169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b2ea-0caf-457c-a8f1-645e13cc349a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.035464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b2ea-0caf-457c-a8f1-645e13cc349a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.035733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b2ea-0caf-457c-a8f1-645e13cc349a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.053592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5296b2ea-0caf-457c-a8f1-645e13cc349a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:30.054852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5296b2ea-0caf-457c-a8f1-645e13cc349a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:30.130530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 493/996] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:30.154053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac198f4-61ab-4b62-ac26-6881b13b925e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.216118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454518c6-c2c8-4087-81fb-a6e8f8a641a7 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.216705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454518c6-c2c8-4087-81fb-a6e8f8a641a7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.216705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454518c6-c2c8-4087-81fb-a6e8f8a641a7 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.218214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 494/997] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:30.218768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.223001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.223284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.223552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.242685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:30.243848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:30.262690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac198f4-61ab-4b62-ac26-6881b13b925e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.282635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.298981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.300002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.300278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.300577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.301228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.302612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:30.315218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:30.316209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:30.317431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 504/998] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:30.320894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 495/999] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:30 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:30.322736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.343522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.426006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda54bb-f934-48ec-8a33-dde8342017e4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.442494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e197af-4a77-433e-8312-ff2770814c91 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.453148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.463022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.463694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.464088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.464385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.465090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] RBAC: Adding query filter params (name=compute) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:30.465397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.466874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.476906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 505/1000] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:30.477772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.477895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.478675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.478956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.479481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.480137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.482595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:30.495856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:30.496883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:30.498355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 496/1001] 10.222.0.45 () {56 vars in 1126 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:30.502824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.541649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.584592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38188010-be61-4faf-b359-f597fa7de5ef None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.610250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b260c50-7cd2-4f90-bf01-4eee538538bd None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.621579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.622238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.622523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.622920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.623737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:30.624070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.625427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.635173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 506/1002] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:30.636336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.644083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.644768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.645047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.645317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.646017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:30.646317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.647691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.657065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 497/1003] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:30.657963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.697644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.746393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4c8569-913c-45d1-aee2-11a393ad3588 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.753546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.758236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f66483-f28c-44f5-b0ab-658f0b24943f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.782928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.783591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.783865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.784131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.784998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] RBAC: Adding query filter params (interface=public, service_id=872719bbb7044fc69c6276a3755b9613, region_id=RegionOne) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:30.785309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.786829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.792734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 507/1004] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/endpoints?service_id=872719bbb7044fc69c6276a3755b9613&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:30.797920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.799643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.800695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.801976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.802426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.803299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:30.803755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.805376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.811961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 498/1005] 10.222.0.45 () {56 vars in 1122 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:30.817337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:30.897501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde42274-4d0c-41a1-ba0e-0e16595c7cc0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.915121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:30.934554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.935228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.935509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.935785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.936572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.937297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab9ae34-218d-48bc-8e36-b094904a25b6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:30.937947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2705fc6-02cf-4ee4-a666-819e836a652d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.946037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 508/1006] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:30 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:30.974263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:30.974950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:30.975227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:30.975495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:30.976251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:30.977733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7be145-03dc-4ee4-89de-77a150f80c98 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:30.996334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 499/1007] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:30 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:31.958565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eff-7a3c-4fa0-b3a9-ab09a6502e82 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:31.959084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eff-7a3c-4fa0-b3a9-ab09a6502e82 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:31.959162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f65eff-7a3c-4fa0-b3a9-ab09a6502e82 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:31.960548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 509/1008] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:31.965898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154c136-e92a-4df2-a4c6-2cc35ca03dd6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:31.966312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154c136-e92a-4df2-a4c6-2cc35ca03dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:31.966517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154c136-e92a-4df2-a4c6-2cc35ca03dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:31.983458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2154c136-e92a-4df2-a4c6-2cc35ca03dd6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:31.984655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2154c136-e92a-4df2-a4c6-2cc35ca03dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:32.001933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:32.068455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 500/1009] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:31 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:32.087001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65eff-7a3c-4fa0-b3a9-ab09a6502e82 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:32.193807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f65eff-7a3c-4fa0-b3a9-ab09a6502e82 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:32.227843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:32.228607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:32.228898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:32.229421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:32.230249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:32.231636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268e441-6c9c-45bf-95ae-1561f0d61ed2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:32.252357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 510/1010] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:32 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:33.166187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aa8ce-1965-4f7e-839b-894e1b1c956a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.166693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aa8ce-1965-4f7e-839b-894e1b1c956a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.166809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27aa8ce-1965-4f7e-839b-894e1b1c956a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.168145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 501/1011] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:33.173292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.173709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.173884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.191834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:33.193067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:33.202092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fdb89-2599-4cbc-ac12-da79568bbe85 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.202378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fdb89-2599-4cbc-ac12-da79568bbe85 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.202654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fdb89-2599-4cbc-ac12-da79568bbe85 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.203994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 502/1012] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:33.208839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.209098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.209337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.226871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:33.227992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:33.232544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:33.280724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 511/1013] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:33 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:33.298318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.304636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 503/1014] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:33 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:33.321080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.404973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:33.405905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7563e-f0a4-4569-ab47-72d23798b283 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.427973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff8a8e-9f9d-4807-8fb5-2c1238dd54fc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.444434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.445959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.446429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.446917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.447768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.450054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.462862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.463660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.463943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.464216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.465441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:33.466316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:33.467297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 15:07:33.468227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:33.469030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.469030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 504/1015] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:33.476996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 512/1016] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:33.477180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.479873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.512219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:33.580952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac775b-1438-43b3-be09-dc2e8a7668ea None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.587722 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74e571-4c48-40a1-8809-7d1c88411a2a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.616290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.616997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.617277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.617778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.618242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.619692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.622414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.623274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.623875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.624162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.624911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:33.625224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.626588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.633228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 513/1017] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:33.638403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.644232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:33.664579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 505/1018] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:07:33 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:33.743056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:33.751391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b43d26-7c65-438e-97f9-9346d0c683c9 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.788815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.789416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.789710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.789975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.790700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:33.791010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.792447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fc6c4-0862-43ce-98b5-9e01dad84bf0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.802727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 514/1019] 10.222.0.45 () {56 vars in 1112 bytes} [Tue May 24 15:07:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:33.803956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:33.912238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a8757-a9f3-4a3e-b025-6ff71a29d128 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:33.948005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:33.948933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:33.949392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:33.949818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:33.950706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:33.952246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7364b-7d1b-4682-a001-bc211d4da7eb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:33.972729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 506/1020] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:34.472214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366310-1c42-4749-8319-dcaeac613847 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:34.472781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366310-1c42-4749-8319-dcaeac613847 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:34.472781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58366310-1c42-4749-8319-dcaeac613847 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:34.474308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 515/1021] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:34.480209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:34.480644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:34.480807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:34.498186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:34.499365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:34.549522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:34.575615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 507/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:34 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:34.597976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58366310-1c42-4749-8319-dcaeac613847 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:34.702070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58366310-1c42-4749-8319-dcaeac613847 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:34.736222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:34.736970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:34.737255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:34.737530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] PATH_INFO: `/v3/services/image` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:34.738179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:34.739554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:34.751135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:34.752153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:34.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 516/1023] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 15:07:34 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:34.758525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:34.862391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c11049-1191-465c-96d5-1866c017aa15 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:34.895934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:34.896630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:34.896913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:34.897184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:34.898423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:34.898767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:34.900157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:34.909622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 508/1024] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:34.911096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:34.937153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:35.012507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3737f-a282-482d-8a52-3a2b3dab6236 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:35.046527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:35.047187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:35.047468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:35.047748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:35.048483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] RBAC: Adding query filter params (type=image) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:35.048792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:35.050171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:35.055316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 517/1025] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:35.060926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:35.110503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:35.167492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a0c22-2ea0-4bfe-85af-6bfc132af9c5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:35.203938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:35.204632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:35.204914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:35.205252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:35.205960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] RBAC: Adding query filter params (interface=public, service_id=a2cc1d59a051435283bb3a5d54573e09, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:35.206271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:35.207648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f36e5c-5675-45e3-b83c-b9b4c086072c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:35.216440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 509/1026] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:07:35 2022] GET /identity/v3/endpoints?service_id=a2cc1d59a051435283bb3a5d54573e09&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:35.221681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:35.311351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:35.324811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b310769f-aa92-4aa8-9ccf-ce94b8aaaf1e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:35.358253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:35.358972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:35.359261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:35.359538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:35.360216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:35.361632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36975cb5-2f73-4bbc-9a70-2b119a5bd40e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:35.376327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 518/1027] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:07:35 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:07:35.936012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843155-d4c1-45a3-8853-26bd6146a718 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:35.936598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843155-d4c1-45a3-8853-26bd6146a718 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:35.936598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c843155-d4c1-45a3-8853-26bd6146a718 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:35.938116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 510/1028] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:35.943168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:35.943543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:35.943741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:35.960287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:35.961509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:36.029905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 519/1029] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:36.055254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c843155-d4c1-45a3-8853-26bd6146a718 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:36.169158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c843155-d4c1-45a3-8853-26bd6146a718 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:36.178602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:36.204484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:36.205198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.205476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.205746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.206906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:36.208379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f07a4-1b34-4b69-b33c-5d6319b2b7ea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:36.210654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 511/1030] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:36.216109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:36.257258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493327e7-81ae-4aaa-a889-26037cd0b03f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.257568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493327e7-81ae-4aaa-a889-26037cd0b03f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.257820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493327e7-81ae-4aaa-a889-26037cd0b03f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.261170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 512/1031] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:36.266216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.266497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.266767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.284693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:36.285864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:36.307201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:36.327450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67105126-cccf-4c57-9796-a7714d71fb0c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:36.357061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 513/1032] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:36.368971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:36.369689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.369969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.370379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.371088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:36.372635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:36.379770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:36.394251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 520/1033] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:07:36 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:36.502183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daab69-6c23-4db4-a9bc-72a06ef7c764 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:36.510279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:36.545324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:36.546146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.546429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.546705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.558596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:36.567008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:36.568160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:36.569036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:36.575452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 514/1034] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:36.576334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:36.685102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3cf1e0-ed42-451b-bdf4-307d05eb56a5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:36.718444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:36.719115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.719394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.719666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.720461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:36.720766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:36.722119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:36.727141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:36.730913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 521/1035] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:36.737241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:36.841858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd7f5d-8292-45f6-b4ea-c2a1dae07f19 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:36.875146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:36.875945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:36.876228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:36.876533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:36.887715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:36.889203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:36.906598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:36.907549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:36.908477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:36.908477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:36.908477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:36.908477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:36.912479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 515/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:36.913772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.017860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d21e2-4513-4701-821a-d9ed437a193b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.052051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.053135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.053619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.054643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.055580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:37.056084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.057687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.081017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 522/1037] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:37.082307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.103293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:37.187774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acdffa-05ed-4b10-915a-100e8c355a36 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.227468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.228476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.228763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.229055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.238114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.239584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:37.253789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:37.254720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:37.260659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 516/1038] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:37.263671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.325700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:37.369041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96cc5bb-6815-408d-a152-349e28e29b32 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.407210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.407880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.408156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.408458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.409356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:37.409658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.411142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073bd9a-ee97-432e-92d5-725018f53e9e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.432791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 523/1039] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:37.434144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.518859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:37.550493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32806f9-496b-48f4-998f-26506ea1ae7c None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.557450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f6a02-33ab-4db6-9784-db0f59f4eea3 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.557741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f6a02-33ab-4db6-9784-db0f59f4eea3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.558020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f6a02-33ab-4db6-9784-db0f59f4eea3 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.559541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 524/1040] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:37.564205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.564519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.564791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.581996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:37.583162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:37.594767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.595442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.595723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.595996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.661416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 525/1041] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:37.671257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.672768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.684616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.737285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:37.755543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:37.779587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 517/1042] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:37.784986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.801197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f48c40-1121-4483-9dd3-908b24e3491a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.837902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.838697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.838976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.839248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] PATH_INFO: `/v3/services/image` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.839929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.841328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:37.853340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:37.854661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:37.855521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 526/1043] 10.222.0.45 () {56 vars in 1105 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:37.860796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.895814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a120-2bda-4a51-bd23-6affb8006d64 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.931206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:37.931927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:37.932203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:37.932496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:37.942862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:37.945082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:37.961511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ccc92-7acb-474d-b280-0efd6ac757fa None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:37.964554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:37.965529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:37.966436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:37.966436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:37.966436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:37.966436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:37.966436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 518/1044] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:37.971288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:37.980850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.003587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.004261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.004574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.004852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.005056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.005562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] RBAC: Adding query filter params (name=image) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:38.005996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.007891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.018366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 527/1045] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:38.019367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.086850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c91316-9df5-4d8c-b13a-cf0e42a12a73 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.123702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.124498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.124778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.125046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.125765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:38.126070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.127701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.129993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d78d93-fbe0-4880-87ee-3ab242c05f9f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.139970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 519/1046] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:38.145048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.168714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.169389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.169679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.169951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.170659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:38.170966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.172359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.177128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 528/1047] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:38.182649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.224820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.245201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.260273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf4fa6-efdc-49f5-a1c9-b62bedfc3ffe None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.290788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3419e22-2459-4422-9ab4-db1d46ca81ce None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.297678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.298465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.298742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.299012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.305790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.307239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:38.322246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:38.323216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:38.324132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 520/1048] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:38.328953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.329101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.329630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.330030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.330311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.331055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.333105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.359843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 529/1049] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:07:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:07:38.439073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd95ab-c4e1-4957-9183-d979838a8ed3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.482253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.482921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.483198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.483470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.484260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:38.484598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.486195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c313b-de1d-4ddb-bb6b-ef1ced46291c None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.504827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 521/1050] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:38.506300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.611596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.619734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bff20-9053-4164-a6c1-9138c6b2b4e7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:38.655026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:38.655689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.655973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.656250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.657115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] RBAC: Adding query filter params (role.id=a0214636f75e478d847025a8f8f9865d, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:38.657429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:38.658878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e5973-bb76-401c-ab27-5ad5aeafdf8d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:38.664881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 530/1051] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:38 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=a0214636f75e478d847025a8f8f9865d => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:38.706473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a50e8-d980-4dcc-826a-0cb58ffca0df None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.706722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a50e8-d980-4dcc-826a-0cb58ffca0df None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.707121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a50e8-d980-4dcc-826a-0cb58ffca0df None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.708655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 522/1052] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:38.714599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:38.714879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:38.715162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:38.733676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:38.734893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:38.810780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 531/1053] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:38.834616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a50e8-d980-4dcc-826a-0cb58ffca0df None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:38.844653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:38.966491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a50e8-d980-4dcc-826a-0cb58ffca0df None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.001609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.002406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.002689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.002960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.013621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:39.021224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:39.022123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:39.027830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 523/1054] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:39.030072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:39.126191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:39.136955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad239fe-5f88-4b3b-97e1-6c1e51954fa9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.174053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.174728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.175006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.175277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.176015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:39.176352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:39.177699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:39.186992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 532/1055] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:39.192651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:39.260199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:39.296916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2ff2-6e7b-4070-82b8-05b14e957ce6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.329429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.330289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.330565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.330833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] PATH_INFO: `/v3/users/923bea6e395740c3af81c954c64a07cc` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.344357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] RBAC: Authorizing `identity:get_user(user_id=923bea6e395740c3af81c954c64a07cc)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:39.345797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:39.347593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 524/1056] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:39 2022] GET /identity/v3/users/923bea6e395740c3af81c954c64a07cc => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:39.353778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:39.451812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48279916-525c-4da7-a5e0-e101ab81ee3f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.485516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.486199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.486479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.486750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.509974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:39.556972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:39.558344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:39.659756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 533/1057] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:39.661094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:39.760068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dced02d-3571-4964-8f46-8cdbf12a0090 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.784010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:39.796779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.797635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.797911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.798179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.799347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:39.800817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613007e3-c91d-4a79-bc46-7a28c5568bce None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:39.807535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 525/1058] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:39 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:39.809067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:39.861522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:39.909187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac71aac7-f7f4-4e9b-8a0d-c50c61911eeb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:39.943292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:39.943945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:39.944225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:39.945061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:39.945905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=3f37a64dc18042b7be01c8387cbee28a, user.id=923bea6e395740c3af81c954c64a07cc) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:39.946216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:39.947645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96294009-8b28-4e24-a73e-242d397e3676 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:39.953282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 534/1059] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:39 2022] GET /identity/v3/role_assignments?user.id=923bea6e395740c3af81c954c64a07cc&scope.project.id=3f37a64dc18042b7be01c8387cbee28a&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:40.590693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67355e-c2c5-4aff-a956-efe468b5d51c None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:40.593404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67355e-c2c5-4aff-a956-efe468b5d51c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:40.593404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67355e-c2c5-4aff-a956-efe468b5d51c None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:40.593404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 526/1060] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:40.598117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab3-c1b1-407d-ac27-f2e1af73f686 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:40.598556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab3-c1b1-407d-ac27-f2e1af73f686 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:40.598647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fdeab3-c1b1-407d-ac27-f2e1af73f686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:40.615663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-72fdeab3-c1b1-407d-ac27-f2e1af73f686 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:40.616896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-72fdeab3-c1b1-407d-ac27-f2e1af73f686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:40.686222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 535/1061] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:40.709458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67355e-c2c5-4aff-a956-efe468b5d51c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:40.813471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67355e-c2c5-4aff-a956-efe468b5d51c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:40.846989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:40.847553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:40.847833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:40.848108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:40.893474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0acba-e975-496a-8d90-c90df3bd520b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:40.893806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0acba-e975-496a-8d90-c90df3bd520b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:40.894050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0acba-e975-496a-8d90-c90df3bd520b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:40.895580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 536/1062] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:40.900268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:40.900661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:40.900949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:40.917233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:40.918416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:40.922150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:40.936055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:40.937854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:40.963681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:40.989809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 537/1063] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:41.012277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.040679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 527/1064] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:07:41.046800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.123227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9518bd5c-b633-4425-9c9f-dcf8cfa063e0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.156292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:41.158179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.159010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.159321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.159561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.169430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323166d-11f6-476b-8199-9760dfb2a8ec None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.172405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:41.182481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:41.183422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:41.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:41.185680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 538/1065] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:41.189987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.208317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.209109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.209391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.209664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.219817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.222362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.233349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:41.247096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:41.248203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:41.249295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:41.249295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:41.249295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:41.249295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:41.249295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 528/1066] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:41.254279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.312899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e2f8b-10e4-45d3-bbd9-eac015329c4b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.347756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.348466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.348757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.349107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.349753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:41.350179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.351958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.359571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419120-dfec-4656-90d4-6f45c9157ebf None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.360537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 539/1067] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:41.365955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.397087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.397674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.397953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.398448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.398925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:41.399225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.400621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.420336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 529/1068] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:41.479979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:41.485433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a8e6c-bfee-45d2-8314-cb3c48a1994d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.523540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.524384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.524673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.524949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.536948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.539075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:41.560473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:41.561516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:41.562414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:41.562414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:41.562414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:41.562414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:41.567841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 540/1069] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:41.569186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.573390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:41.682631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe0e53-03fa-4ff5-9e4c-837b9390325f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.717159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.717832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.718114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.718388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.719161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:41.719480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.720918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.734475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 530/1070] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:41.740076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.841195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5bf7a9-a0af-4d84-8c02-714a65008e38 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:41.873346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:41.874132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:41.874412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:41.874683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:41.882477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:41.883932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:41.899575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:41.900823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:41.901784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 541/1071] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:41.906549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:41.984188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.021260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da546338-5728-444f-8a3d-a79853f1b5be None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.054102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.054762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.055039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.055309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.056131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:42.056470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.057898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595bde0-49bf-4d06-a62d-abf785d45c67 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.077683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 531/1072] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:07:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:42.083199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.096511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.140772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314816-a146-40ab-8891-13213cfaa85f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.141058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314816-a146-40ab-8891-13213cfaa85f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.141332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04314816-a146-40ab-8891-13213cfaa85f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.142701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 532/1073] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:07:42.147584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.147870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.148150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.165026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:42.166194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:42.183531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ac8d2-baef-4302-8121-0311b95c835c None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.217971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.218758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.219036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.219309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.230329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.231788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.233629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 533/1074] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:42.249297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:42.250342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:42.251236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:42.251236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:42.251236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:42.251236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:42.251236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 542/1075] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:42.256131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.256260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.308763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.316905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.355253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60f632-eb70-40fb-b0c4-dbe88488de7a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.355618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79756222-6151-45b1-93ac-4a0dd090f955 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.387296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.387710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.387943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.388219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.388517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.388612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.388799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.389074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.389228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:42.389529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.390904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.399646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:42.402384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 534/1076] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:42.407122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:42.408271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:42.409182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:42.409811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.409894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 543/1077] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:42.414538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.506736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed5cc5-ccaf-4785-ba56-ef1f593e185b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.511033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85c80c-4358-47f5-8403-05047d52c2eb None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.518730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.533391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.541254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.542037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.542318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.542588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.545362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.546011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.546290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.546559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.547295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:42.547595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.549000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.549144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.550625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.557313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 544/1078] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:42.563082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:42.564912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:42.565909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:42.566807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 535/1079] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:42.572132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.661429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12308c0-15c6-4a49-8bc7-057ef9f259e8 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.669486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fde479-6798-4c3d-8118-122333ed0f22 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.693891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.695233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.695514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.695789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] PATH_INFO: `/v3/users/923bea6e395740c3af81c954c64a07cc` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.697043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] RBAC: Authorizing `identity:get_user(user_id=923bea6e395740c3af81c954c64a07cc)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.698507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.700128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 545/1080] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/users/923bea6e395740c3af81c954c64a07cc => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:42.703338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.703993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.704268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.704569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.705486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:42.705759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.705860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.707215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.719427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 536/1081] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:42.725187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.755264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.759495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.807380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbad01-bb62-4af6-870a-2e4f6245879d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.825947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5971e-f7c4-4d17-a875-ed8718ff0ce9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.841265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.842117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.842392 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.842659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.843814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.845269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.846950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 546/1082] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:42 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:42.852697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:42.859818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.860731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.861010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.861283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/a0214636f75e478d847025a8f8f9865d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:42.863495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=aa1fb96699054e5c94fcc188fd9535be, role_id=a0214636f75e478d847025a8f8f9865d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:42.864965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff9869-7a27-4519-9e3a-75fe42e01091 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:42.878142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 537/1083] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:42 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/aa1fb96699054e5c94fcc188fd9535be/roles/a0214636f75e478d847025a8f8f9865d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:07:42.961841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b777-af5c-4b9f-82ae-2897ecbe5998 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:42.982953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:42.998179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:42.999083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:42.999363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:42.999635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a/users/923bea6e395740c3af81c954c64a07cc/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:43.001705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] RBAC: Authorizing `identity:create_grant(project_id=3f37a64dc18042b7be01c8387cbee28a, user_id=923bea6e395740c3af81c954c64a07cc, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:43.003174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be73067-0c98-4786-bd65-e73246abc34c None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:43.018427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 547/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:42 2022] PUT /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a/users/923bea6e395740c3af81c954c64a07cc/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:07:45.089429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c533f-d490-42dd-af3c-7744689745c9 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.089914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c533f-d490-42dd-af3c-7744689745c9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.090031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c533f-d490-42dd-af3c-7744689745c9 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.091341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 538/1085] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:45.096535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.096919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.097098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.114163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:45.115341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:45.176348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f14-ede3-48d5-9bcf-9968bf42d1ba None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.176631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f14-ede3-48d5-9bcf-9968bf42d1ba None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.176906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea3f14-ede3-48d5-9bcf-9968bf42d1ba None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.178263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 539/1086] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:45.182020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 548/1087] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:45.183879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.184159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.184465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.200746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:45.201905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:45.203754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.270512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 540/1088] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:45.294325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.300381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4edfa6-7a98-4209-9e06-df757456a6fa None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.311075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.317316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.336690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.337445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.337727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.338001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.348125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:45.357173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:45.358275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:45.359513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 549/1089] 10.222.0.45 () {56 vars in 1120 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:45.364111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.392814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0a76e-3823-468a-9a89-37d8ecc6281e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.426724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.427508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.427791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.428069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.437990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:07:45.445583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:45.446637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:45.447471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 541/1090] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:45.452670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.460656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb9483-51cf-4669-9dcf-933dbc00d68e None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.494831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.495489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.495767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.496038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.496812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:45.497116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.498476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.506337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 550/1091] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:45.511614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.536080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.542882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.555459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061f5c2-fb8b-437e-b326-cfba2bb8d5f6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.589295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.589896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.590180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.590457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.591202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:45.591510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.592938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.600912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 542/1092] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:45.611678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.615751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6b693d-2980-4bb3-a8d1-5d4e0d112015 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.649147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.649936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.650215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.650485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.661834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.663398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:45.680356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:45.681484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:45.682676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:45.682676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:45.682676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:45.682676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:45.686268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 551/1093] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:07:45.687009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.708980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5b123-50cd-4f17-b969-ff1057269d60 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.742324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.743286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.743635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.743916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] PATH_INFO: `/v3/users/923bea6e395740c3af81c954c64a07cc` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.745149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] RBAC: Authorizing `identity:get_user(user_id=923bea6e395740c3af81c954c64a07cc)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.746585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.748209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 543/1094] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/users/923bea6e395740c3af81c954c64a07cc => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:45.753671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.768475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.775362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.794873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd2f83-2b30-40c5-84a8-160d24a1a406 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.832777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.833442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.833725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.834003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.834785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:45.835589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.836991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.853037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 552/1095] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:45.854414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:45.856961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9c681-f099-4074-aa03-cef6920c3d47 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.890121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.890766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.891047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.891319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.952412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e95ba-656e-45b4-8262-1397c393f349 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:45.959548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.961215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:45.975292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:45.986269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:45.987068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:45.987382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:45.987660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:45.995257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:45.996730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:07:46.012054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:46.018314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 553/1096] 10.222.0.45 () {56 vars in 1093 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:46.019371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.022554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.061898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 544/1097] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:07:46.067729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.122015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55f1d9-1c0d-41b8-9249-60738d119c55 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.156254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.157051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.157330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.157604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.158432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:46.158734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.160335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.167848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ad75c-9a38-4086-b5bd-b0acef1411ff None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.178042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 554/1098] 10.222.0.45 () {56 vars in 1133 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:46.181429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.184202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.203297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.204147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.204454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.204727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.205973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.207413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.209304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 545/1099] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:46.215428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.247058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.285840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3765e38-e5c2-4219-bf99-9e58dce6bc12 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.316647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1376c-7c95-4521-9af0-3bcb9ad261a0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.319860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.320538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.320820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.321088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.351307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.351951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.352228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.352531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.353346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=3f37a64dc18042b7be01c8387cbee28a, user.id=923bea6e395740c3af81c954c64a07cc) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:46.353657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.355345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.361056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 546/1100] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/role_assignments?user.id=923bea6e395740c3af81c954c64a07cc&scope.project.id=3f37a64dc18042b7be01c8387cbee28a&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:46.391316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.392795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.456237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.498571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 555/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:07:46.504603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.555212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.610099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26185a07-47f8-4e2c-931a-4a255c9e01e7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.643898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.644642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.644923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.645196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.656020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.657518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:07:46.674695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:46.675708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:46.676619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:07:46.676619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:07:46.676619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:07:46.676619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:46.680457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 547/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:46.682323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.780248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ca03e-7856-4258-bc54-4be9617e4b45 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.813797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.814460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.814739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.815010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.815727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:46.816032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.817435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.828977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 556/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:46.834482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:46.928375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:46.935220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5b271-ca77-4176-9000-84e0e59e6b39 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:46.969377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:46.970168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:46.970450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:46.970723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:46.977912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:46.979636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:07:46.993082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:07:46.994065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:07:46.995005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:07:46.996446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 548/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:07:47.000017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:47.025093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:47.102750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff70d5c-c2cd-4118-b43b-49e3b5f3cfda None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:47.135700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:47.136384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:47.136667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:47.136941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:47.137724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:47.138035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:47.139466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b6b614-3a28-4401-b9f4-0a2d10b6f0ef None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:47.151838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 557/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:07:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:47.157969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:47.255725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd98e6c-d565-433f-a808-f7b21fa4be32 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:47.289812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:47.290453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:47.290735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:47.291006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:47.291826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] RBAC: Adding query filter params (role.id=a0214636f75e478d847025a8f8f9865d, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=aa1fb96699054e5c94fcc188fd9535be) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:47.292133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:47.293598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9503d01b-7f5d-4184-9955-3b634363309b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:47.299460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 549/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:47 2022] GET /identity/v3/role_assignments?user.id=aa1fb96699054e5c94fcc188fd9535be&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=a0214636f75e478d847025a8f8f9865d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:48.564197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacedfa3-497a-45ac-a9c3-b0268abeb6dd None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:48.564786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacedfa3-497a-45ac-a9c3-b0268abeb6dd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:48.564786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacedfa3-497a-45ac-a9c3-b0268abeb6dd None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:48.566113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 558/1107] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:48.571698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:48.572109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:48.572252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:48.589395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:48.590597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:48.604443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:48.656017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 550/1108] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:48.680145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacedfa3-497a-45ac-a9c3-b0268abeb6dd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:48.759937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:48.779387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacedfa3-497a-45ac-a9c3-b0268abeb6dd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:48.812242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:48.813046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:48.813324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:48.813595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:48.814797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:48.816167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f0e8df-3f97-409c-be60-54b5d2d36385 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:48.817803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 559/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:07:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:48.823911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:48.919691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2218cc-b400-4111-b705-fddda94786c8 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:48.953164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:48.953820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:48.954104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:48.954379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:48.955105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:48.956572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d02e4-a5cc-4694-a0a4-6125cb37470f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:48.975839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 551/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:07:48 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:07:51.176069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51bf8-b7c0-4750-a01b-c2aba09a3951 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:51.176615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51bf8-b7c0-4750-a01b-c2aba09a3951 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:51.176715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c51bf8-b7c0-4750-a01b-c2aba09a3951 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:51.178025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 560/1111] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:51.183704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:51.184146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:51.184240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:51.196155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:51.201852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:51.203179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:51.267754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 552/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:07:51.290892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c51bf8-b7c0-4750-a01b-c2aba09a3951 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:51.389851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c51bf8-b7c0-4750-a01b-c2aba09a3951 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:51.421924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:51.422760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:51.423039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:51.423310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] PATH_INFO: `/v3/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:51.424803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] RBAC: Authorizing `identity:get_role(role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:51.426182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:51.427527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 561/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:51 2022] GET /identity/v3/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:51.433685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:51.528205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecbd4c-c091-4d1b-a4dc-46728069193d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:51.560477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:51.561327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:51.561602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:51.561868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] PATH_INFO: `/v3/users/25b8660415db4e4885947a6d4aa2626f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:51.564605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:51.575436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] RBAC: Authorizing `identity:get_user(user_id=25b8660415db4e4885947a6d4aa2626f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:51.576923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:51.578546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 553/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:51 2022] GET /identity/v3/users/25b8660415db4e4885947a6d4aa2626f => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:51.584151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:51.602861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:51.681764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997571ad-620c-4231-a14e-2eabe6543e66 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:51.713407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:51.714038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:51.714319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:51.714593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:51.785068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:51.786512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:51.805373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:51.882654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 562/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:07:51.888825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:51.985514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f459af-ce80-41a6-98d2-24162b6f6b42 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:52.020499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:52.021354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:52.021631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:52.021901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:52.023595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:52.025077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24786c45-a60c-436b-b764-327a9ab7b11d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:52.026770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 554/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:51 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:52.032948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:52.128450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb69586-283d-4183-b227-ad042b920248 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:52.144689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:52.161301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:52.161942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:52.162220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:52.162492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:52.163979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=3f37a64dc18042b7be01c8387cbee28a, user.id=25b8660415db4e4885947a6d4aa2626f) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:52.164284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:52.165737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5424c1a6-4c10-4dc2-8b18-24ef52145f2f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:52.170957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 563/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:52 2022] GET /identity/v3/role_assignments?user.id=25b8660415db4e4885947a6d4aa2626f&scope.project.id=3f37a64dc18042b7be01c8387cbee28a&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:54.307943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6179fc9-9df2-48dc-815e-c8ee007fa19f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.308505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6179fc9-9df2-48dc-815e-c8ee007fa19f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.308505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6179fc9-9df2-48dc-815e-c8ee007fa19f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.309871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 555/1118] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:54.315177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.315573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.315717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.332002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:54.333152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:54.397705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 564/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:54.420705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6179fc9-9df2-48dc-815e-c8ee007fa19f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:54.481294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:54.518494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6179fc9-9df2-48dc-815e-c8ee007fa19f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:54.551345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:54.552176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.552500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.552769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] PATH_INFO: `/v3/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.554248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] RBAC: Authorizing `identity:get_role(role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:54.555624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:54.556988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 556/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:54 2022] GET /identity/v3/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:54.562615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:54.657585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a8f51-b621-425c-8bbf-33ab9152bf97 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:54.669940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:54.691570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:54.692485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.692770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.693050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] PATH_INFO: `/v3/users/25b8660415db4e4885947a6d4aa2626f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.694723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] RBAC: Authorizing `identity:get_user(user_id=25b8660415db4e4885947a6d4aa2626f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:54.696154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:54.697823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 565/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:54 2022] GET /identity/v3/users/25b8660415db4e4885947a6d4aa2626f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:54.704327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:54.800550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5293506-0ada-47c5-bf96-7606e5c316b6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:54.828122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:54.833823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:54.835262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.835540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.835810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.836989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:54.838434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97f0d8-4693-40d1-845d-7daaf38e78b8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:54.840110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 557/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:54 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:54.846142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:54.941650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ccbf3-3738-4a27-9c45-57ba00da9f60 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:54.974754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:54.975651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:54.975926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:54.976193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a/users/25b8660415db4e4885947a6d4aa2626f/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:54.978375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] RBAC: Authorizing `identity:create_grant(project_id=3f37a64dc18042b7be01c8387cbee28a, user_id=25b8660415db4e4885947a6d4aa2626f, role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:54.980056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033bd82-474d-450d-8e52-da1797759342 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:54.992635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 566/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:07:54 2022] PUT /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a/users/25b8660415db4e4885947a6d4aa2626f/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:07:57.125363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02a32-6fa9-46f1-88cc-363ac54a41dd None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.125879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02a32-6fa9-46f1-88cc-363ac54a41dd None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.125971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a02a32-6fa9-46f1-88cc-363ac54a41dd None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.127268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 558/1124] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:07:57.132933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.133323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.133481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.149789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:07:57.150948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:07:57.184663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:57.217726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 567/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:07:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:07:57.242034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02a32-6fa9-46f1-88cc-363ac54a41dd None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:57.341842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a02a32-6fa9-46f1-88cc-363ac54a41dd None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:57.376469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:57.377314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.377591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.377860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] PATH_INFO: `/v3/roles/5545766f03fb4bffac3e25eab0e2dee8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.379301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] RBAC: Authorizing `identity:get_role(role_id=5545766f03fb4bffac3e25eab0e2dee8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:57.380714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:57.382828 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 559/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:57 2022] GET /identity/v3/roles/5545766f03fb4bffac3e25eab0e2dee8 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:07:57.388651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:57.482039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d568b5-c396-4951-a661-2aa7b1031cec None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:57.514303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:57.515159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.515434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.515705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] PATH_INFO: `/v3/users/25b8660415db4e4885947a6d4aa2626f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.516886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] RBAC: Authorizing `identity:get_user(user_id=25b8660415db4e4885947a6d4aa2626f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:57.518301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:57.519869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 568/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:07:57 2022] GET /identity/v3/users/25b8660415db4e4885947a6d4aa2626f => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:57.525379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:57.591277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:57.625690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42438d3d-00b2-4ed1-a6ed-df451ee20c15 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:57.658568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:57.659213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.659494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.659766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.724055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:57.725550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:57.786150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:57.821078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 560/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:07:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:07:57.827034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:57.865721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:07:57.923530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24770055-2f68-4370-9b40-7fbe46396083 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:57.955926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:57.956795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:57.957074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:57.957343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] PATH_INFO: `/v3/projects/3f37a64dc18042b7be01c8387cbee28a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:57.958498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] RBAC: Authorizing `identity:get_project(project_id=3f37a64dc18042b7be01c8387cbee28a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:57.959948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c66df0-cfb7-4738-ac0b-576e29abe2f7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:57.961636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 569/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:07:57 2022] GET /identity/v3/projects/3f37a64dc18042b7be01c8387cbee28a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:07:57.967493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:07:58.061969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73d86-98e4-431a-bc69-2b46310902c2 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:07:58.095047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:07:58.095719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:07:58.095996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:07:58.096265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:07:58.097107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] RBAC: Adding query filter params (role.id=5545766f03fb4bffac3e25eab0e2dee8, scope.project.id=3f37a64dc18042b7be01c8387cbee28a, user.id=25b8660415db4e4885947a6d4aa2626f) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:07:58.097413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:07:58.098840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788258f-e9ae-4368-966b-841cc84ff1af None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:07:58.104756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 561/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:07:57 2022] GET /identity/v3/role_assignments?user.id=25b8660415db4e4885947a6d4aa2626f&scope.project.id=3f37a64dc18042b7be01c8387cbee28a&role.id=5545766f03fb4bffac3e25eab0e2dee8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:00.305020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4040-a756-41a8-8e88-568ca0783e7c None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:00.305538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4040-a756-41a8-8e88-568ca0783e7c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:00.305654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d4040-a756-41a8-8e88-568ca0783e7c None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:00.306961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 570/1131] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:00.312127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:00.312562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:00.312723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:00.329140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:00.330253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:00.354356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:00.396337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 562/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:00.420403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d4040-a756-41a8-8e88-568ca0783e7c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:00.518632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d4040-a756-41a8-8e88-568ca0783e7c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:00.525765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:00.552625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:00.553434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:00.553719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:00.553995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:00.555156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:00.556626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ee7a2-c9e9-4ba4-b298-440289ee028a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:00.558216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 571/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:08:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:00.563761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:00.659607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f308e-4e19-471f-9259-9b392e15b8b4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:00.692083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:00.692753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:00.693038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:00.693311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:00.694761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:00.696197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203be480-c17f-4f29-a90c-418cfb1dee98 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:00.720956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 563/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:08:00 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:08:02.908198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db90c-fa1b-4115-9195-c2e4a03aeab3 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:02.908791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db90c-fa1b-4115-9195-c2e4a03aeab3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:02.908791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115db90c-fa1b-4115-9195-c2e4a03aeab3 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:02.910119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 572/1135] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:02.915374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:02.915746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:02.915928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:02.932148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:02.933323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:02.940267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:02.998317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 564/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:03.023382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db90c-fa1b-4115-9195-c2e4a03aeab3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:03.120678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115db90c-fa1b-4115-9195-c2e4a03aeab3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:03.153612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:03.154404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:03.154686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:03.154958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:03.156115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:03.157567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f153199-f22a-4e99-9263-66a063ba1639 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:03.159233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 573/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:08:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:03.164845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:03.261686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011efa9-0b8d-415a-b850-0fc47e610c47 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:03.294319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:03.294972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:03.295568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:03.295847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:03.296592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:03.298034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4edb3b-605b-47af-a5aa-2482feb33fbe None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:03.313508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:03.321634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 565/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:08:03 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:08:05.531152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a087f-defe-4ef0-820b-9e905d972e4c None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:05.531665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a087f-defe-4ef0-820b-9e905d972e4c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:05.531751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a087f-defe-4ef0-820b-9e905d972e4c None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:05.533129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 574/1139] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:05.538856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:05.539292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:05.539386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:05.555578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:05.556763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:05.625715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 566/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:05.649058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a087f-defe-4ef0-820b-9e905d972e4c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:05.691980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:05.747450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a087f-defe-4ef0-820b-9e905d972e4c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:05.780529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:05.781350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:05.781577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:05.781846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:05.792412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:05.799932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:05.801063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:05.802089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 575/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:05.806970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:05.902886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf1280-727d-4ee4-8e49-e7e30d050a83 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:05.928293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:05.936028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:05.936722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:05.937003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:05.937277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:05.938014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:05.938319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:05.939702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:05.948018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 567/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:05.953594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:06.049163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ce0dc-bbd3-426c-bdf7-4fd9437bedaf None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:06.050596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:06.082773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:06.083639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:06.083917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:06.084769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] PATH_INFO: `/v3/users/5ab1dbf957b64d789d35bb735ab8dd52` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:06.097697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] RBAC: Authorizing `identity:get_user(user_id=5ab1dbf957b64d789d35bb735ab8dd52)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:06.099146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:06.100858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 576/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:05 2022] GET /identity/v3/users/5ab1dbf957b64d789d35bb735ab8dd52 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:06.106825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:06.202664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7549e78-450f-4598-a195-90a6720fc765 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:06.234126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:06.234751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:06.235028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:06.235297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:06.289265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:06.299544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:06.300988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:06.394648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 568/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:08:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:08:06.400293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:06.495178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7330a43-7d19-4341-aa6b-46b8c29c5bfc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:06.527442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:06.528277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:06.528588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:06.528862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] PATH_INFO: `/v3/projects/5f1840a644f141adab5f2871bff7e509` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:06.530006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] RBAC: Authorizing `identity:get_project(project_id=5f1840a644f141adab5f2871bff7e509)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:06.531427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b131f6-a03c-493f-8902-c098ea78c253 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:06.533112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 577/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:06 2022] GET /identity/v3/projects/5f1840a644f141adab5f2871bff7e509 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:06.538766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:06.626316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:06.635824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab2bdc6-cd98-4890-9334-65c97c0cdd79 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:06.668280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:06.668948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:06.669229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:06.669503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:06.670320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=5f1840a644f141adab5f2871bff7e509, user.id=5ab1dbf957b64d789d35bb735ab8dd52) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:06.670629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:06.672058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82402003-7b05-4c23-911c-7110fa15ae86 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:06.677423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 569/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:08:06 2022] GET /identity/v3/role_assignments?user.id=5ab1dbf957b64d789d35bb735ab8dd52&scope.project.id=5f1840a644f141adab5f2871bff7e509&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:08.899387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50b40a-3adb-4c9f-8a31-9d739e1072f1 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:08.899945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50b40a-3adb-4c9f-8a31-9d739e1072f1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:08.899945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50b40a-3adb-4c9f-8a31-9d739e1072f1 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:08.901324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 578/1147] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:08.906606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:08.907007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:08.907156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:08.923541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:08.924753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:08.989524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 570/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:09.012051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50b40a-3adb-4c9f-8a31-9d739e1072f1 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:09.039654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:09.113808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50b40a-3adb-4c9f-8a31-9d739e1072f1 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:09.146582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:09.147357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:09.147640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:09.147909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:09.157736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:09.165772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:09.166886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:09.168058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 579/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:09.172905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:09.254515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:09.270318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d35a3-2f70-43ed-90cd-1a5b025bad53 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:09.303695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:09.304374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:09.304659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:09.304932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:09.305671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:09.305977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:09.307355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:09.315126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 571/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:09.320839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:09.405682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:09.418006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29e35b-802f-43bf-884f-236e2c3590e6 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:09.451252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:09.452103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:09.452409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:09.452682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] PATH_INFO: `/v3/users/5ab1dbf957b64d789d35bb735ab8dd52` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:09.453848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] RBAC: Authorizing `identity:get_user(user_id=5ab1dbf957b64d789d35bb735ab8dd52)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:09.455272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:09.456896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 580/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:09 2022] GET /identity/v3/users/5ab1dbf957b64d789d35bb735ab8dd52 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:09.462961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:09.560098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b2c4fd-3a1a-4d80-9e22-c7ff25cbf070 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:09.592971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:09.593816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:09.594092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:09.594359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] PATH_INFO: `/v3/projects/5f1840a644f141adab5f2871bff7e509` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:09.595541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] RBAC: Authorizing `identity:get_project(project_id=5f1840a644f141adab5f2871bff7e509)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:09.597006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226fd85-e98b-4c32-a5b7-6b9927cea28a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:09.598648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 572/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:09 2022] GET /identity/v3/projects/5f1840a644f141adab5f2871bff7e509 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:09.604520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:09.699862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493e1ec-340f-4f76-a834-6071ee88245b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:09.733195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:09.734078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:09.734361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:09.734636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] PATH_INFO: `/v3/projects/5f1840a644f141adab5f2871bff7e509/users/5ab1dbf957b64d789d35bb735ab8dd52/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:09.736599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] RBAC: Authorizing `identity:create_grant(project_id=5f1840a644f141adab5f2871bff7e509, user_id=5ab1dbf957b64d789d35bb735ab8dd52, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:09.738014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855bb05a-1ae5-47de-9f8f-cac14dc4b9ce None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:09.751695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 581/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:08:09 2022] PUT /identity/v3/projects/5f1840a644f141adab5f2871bff7e509/users/5ab1dbf957b64d789d35bb735ab8dd52/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:08:11.894993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f575-7356-4079-86d9-8a87a70b6cc1 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:11.895545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f575-7356-4079-86d9-8a87a70b6cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:11.895545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67f575-7356-4079-86d9-8a87a70b6cc1 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:11.896922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 573/1154] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:11.902257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:11.902665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:11.902809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:11.920065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:11.921280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:11.932157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:11.985884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 582/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:12.009382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f575-7356-4079-86d9-8a87a70b6cc1 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.025027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:12.110230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67f575-7356-4079-86d9-8a87a70b6cc1 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.143021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:12.143800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:12.144080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:12.144379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:12.154612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:12.162498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:12.163590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:08:12.164497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 574/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:12.169701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.265062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfbf7d-56a1-4272-9f75-edc7e2056c99 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.297935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:12.298584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:12.298866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:12.299165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:12.299903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:12.300208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:12.301627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:12.309433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 583/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:12.315076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.386264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:12.411336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa6e9d1-e7da-4e5f-b1ed-17d6901c50b3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.444698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:12.445544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:12.445822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:12.446090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] PATH_INFO: `/v3/users/5ab1dbf957b64d789d35bb735ab8dd52` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:12.447252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] RBAC: Authorizing `identity:get_user(user_id=5ab1dbf957b64d789d35bb735ab8dd52)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:12.448737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:12.450357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 575/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/users/5ab1dbf957b64d789d35bb735ab8dd52 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:12.456210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.467696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:12.552622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225350a-a479-4861-b02c-801b5c1bc890 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.584859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:12.585483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:12.585761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:12.586029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:12.649617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:12.651050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:12.660017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:12.745838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 584/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:08:12.751767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.848970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b78704-9632-4bc6-97b2-ca6b4d44f26b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.881630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:12.882463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:12.882743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:12.883012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] PATH_INFO: `/v3/projects/5f1840a644f141adab5f2871bff7e509` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:12.884167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] RBAC: Authorizing `identity:get_project(project_id=5f1840a644f141adab5f2871bff7e509)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:12.885656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d0c7f-d044-4e70-8e90-2ee50ae1a96e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:12.887351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 576/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/projects/5f1840a644f141adab5f2871bff7e509 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:12.893346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:12.989195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7455215-9064-4873-addf-b00adc9c8c5d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:12.998179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:13.022651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:13.023293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:13.023576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:13.023853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:13.024701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=5f1840a644f141adab5f2871bff7e509, user.id=5ab1dbf957b64d789d35bb735ab8dd52) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:13.025014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:13.026448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb9566d-f1b7-4246-89c8-c63393186232 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:13.032089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 585/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:08:12 2022] GET /identity/v3/role_assignments?user.id=5ab1dbf957b64d789d35bb735ab8dd52&scope.project.id=5f1840a644f141adab5f2871bff7e509&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:08:15.215118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81926-e448-490f-a021-adce39bbc9db None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:15.215624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81926-e448-490f-a021-adce39bbc9db None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:15.215733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d81926-e448-490f-a021-adce39bbc9db None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:15.217068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 577/1162] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:15.222359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:15.222721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:15.222909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:15.239941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:15.241143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:15.304912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 586/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:15.326714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81926-e448-490f-a021-adce39bbc9db None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:15.339206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:15.427039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d81926-e448-490f-a021-adce39bbc9db None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:15.460814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:15.461439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:15.461722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:15.461996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:15.526676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:15.528119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:15.539022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:15.622320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 578/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:08:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:08:15.628240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:15.722013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75f32e-1fd4-4a5c-ba9a-eb5c82320867 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:15.723477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:15.755014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:15.756331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:15.756605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:15.756873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:15.763266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:15.764872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-281a4f62-2dba-4bb9-9672-14e8344d2baa None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:15.779125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:15.780329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:15.781380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 587/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:08:15 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:15.786266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:15.881490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c48e0-e1f4-427d-b306-377c93c770c5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:15.891595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:15.915564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:15.916212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:15.916519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:15.916815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:15.917504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:15.917809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:15.919200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308f962-19ba-4b49-916e-57fb7511524e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:15.926411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 579/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Tue May 24 15:08:15 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:18.079279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257937-a4e3-4070-b22e-c798c9539f8d None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:18.079734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257937-a4e3-4070-b22e-c798c9539f8d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:18.079837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41257937-a4e3-4070-b22e-c798c9539f8d None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:18.081210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 588/1167] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:18.087138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a9a4-ae7e-4016-9227-bb489d7ad433 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:18.087541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a9a4-ae7e-4016-9227-bb489d7ad433 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:18.087687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a9a4-ae7e-4016-9227-bb489d7ad433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:18.104265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bf01a9a4-ae7e-4016-9227-bb489d7ad433 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:18.105449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bf01a9a4-ae7e-4016-9227-bb489d7ad433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:18.171144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 580/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:18.194290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257937-a4e3-4070-b22e-c798c9539f8d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:18.290786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41257937-a4e3-4070-b22e-c798c9539f8d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:18.323098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:18.323682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:18.323958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:18.324226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:18.324967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:18.326307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92383527-a981-47fc-bdbe-4afc02b7e318 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:18.341900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 589/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Tue May 24 15:08:18 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:08:20.524244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c525-821a-4b53-aa9c-6f28d16e7189 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:20.524794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c525-821a-4b53-aa9c-6f28d16e7189 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:20.524895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c525-821a-4b53-aa9c-6f28d16e7189 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:20.526188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 581/1170] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:20.531310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:20.531747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:20.531871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:20.548222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:20.549434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:20.565687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:20.616362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 590/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:20.639049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3c525-821a-4b53-aa9c-6f28d16e7189 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:20.737696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3c525-821a-4b53-aa9c-6f28d16e7189 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:20.740006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:20.770114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:20.770955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:20.771746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:20.772018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] PATH_INFO: `/v3/domains/6a538a1856f249bcaf97d07c0d630863` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:20.782130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a538a1856f249bcaf97d07c0d630863)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:20.783525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f3be3-bdaf-4b4e-8931-bb657c507dc5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:20.785146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 582/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Tue May 24 15:08:20 2022] GET /identity/v3/domains/6a538a1856f249bcaf97d07c0d630863 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:20.790903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:20.885716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1711f-9b00-4d4d-b0ab-13c856714c93 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:20.917400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:20.918088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:20.918363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:20.918627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:20.919328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:20.920789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c08bee-c58e-4c62-a082-e66b666a4d65 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:20.945549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 591/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:08:20 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:08:23.209655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5001bd3-a203-40fd-8e47-e342a907a97b None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:23.210198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5001bd3-a203-40fd-8e47-e342a907a97b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:23.210198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5001bd3-a203-40fd-8e47-e342a907a97b None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:23.211534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 583/1174] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:23.216892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:23.217301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:23.217446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:23.223251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:23.234626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:23.235794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:23.298937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 592/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:23.323613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5001bd3-a203-40fd-8e47-e342a907a97b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:23.422072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5001bd3-a203-40fd-8e47-e342a907a97b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:23.454623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:23.455461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:23.455741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:23.456021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] PATH_INFO: `/v3/domains/6a538a1856f249bcaf97d07c0d630863` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:23.457190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=6a538a1856f249bcaf97d07c0d630863)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:23.458668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3431f0-3fa5-4cfc-a5c3-b0440bfc4fd5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:23.460242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 584/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Tue May 24 15:08:23 2022] GET /identity/v3/domains/6a538a1856f249bcaf97d07c0d630863 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:23.465630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:23.560733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381aa81-8c6c-49c3-953e-ecc3c51ab0f1 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:23.592950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:23.593553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:23.593829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:23.594096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:23.594805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:23.596225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8ad6b-f73f-49ea-bb75-5d7a52ddefb0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:23.602692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:23.616537 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 593/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:08:23 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:08:25.826554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d77c-67bb-4ad5-ad7a-85f10125b231 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:25.827114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d77c-67bb-4ad5-ad7a-85f10125b231 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:25.827114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae4d77c-67bb-4ad5-ad7a-85f10125b231 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:25.828488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 585/1178] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:25.833728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:25.834119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:25.834284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:25.850554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:25.851725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:25.915984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 594/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:25.939355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4d77c-67bb-4ad5-ad7a-85f10125b231 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:25.971079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:26.040146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae4d77c-67bb-4ad5-ad7a-85f10125b231 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.073012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.073790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.074069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.074340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.084877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:26.092276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:26.093425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:08:26.094685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 586/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:26.099843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:26.195405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d23457-93d9-4afd-bb60-90fa0057868f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.214462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:26.229676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.230330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.230608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.230881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.231643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:26.231922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:26.233312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:26.241692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 595/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:26.247261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:26.335088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:26.343985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ae026-0b1a-47bd-8eae-0c53d831ab9b None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.377539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.378382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.378664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.378937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] PATH_INFO: `/v3/users/ef8e2e59563e46449a407413c05ebd84` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.391444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] RBAC: Authorizing `identity:get_user(user_id=ef8e2e59563e46449a407413c05ebd84)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:26.392948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:26.394566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 587/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:26 2022] GET /identity/v3/users/ef8e2e59563e46449a407413c05ebd84 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:26.400145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:26.496547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fd720-4961-4e32-ba44-73788985dbec None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.528619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.529256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.529540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.529816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.576421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:26.595278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:26.596713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:26.694096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 596/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:08:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:08:26.699886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:26.795145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f08e99-7aa0-4db1-a23a-3daeb8f9f443 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.828038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.828911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.829187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.829456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] PATH_INFO: `/v3/projects/8b735d8de1484445a6f71827eb0cc943` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.830614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] RBAC: Authorizing `identity:get_project(project_id=8b735d8de1484445a6f71827eb0cc943)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:26.832054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8536450-e077-41e9-9c4e-0a8a70cffaea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:26.833753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 588/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:26 2022] GET /identity/v3/projects/8b735d8de1484445a6f71827eb0cc943 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:26.839423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:26.924026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:26.942286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1bc8e6-22fe-4f10-87fd-b41b9cd78603 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:26.975762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:26.976436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:26.976715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:26.976987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:26.977808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=8b735d8de1484445a6f71827eb0cc943, user.id=ef8e2e59563e46449a407413c05ebd84) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:26.978116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:26.979812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10727de-d5e0-4ba5-bb3c-75e69df15f0a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:26.985097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 597/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:08:26 2022] GET /identity/v3/role_assignments?user.id=ef8e2e59563e46449a407413c05ebd84&scope.project.id=8b735d8de1484445a6f71827eb0cc943&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:29.154418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07605-9195-4304-adf5-75168d0fcae5 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.154932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07605-9195-4304-adf5-75168d0fcae5 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.155031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07605-9195-4304-adf5-75168d0fcae5 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.156391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 589/1186] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:29.161612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.161999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.162153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.178399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:29.179572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:29.243100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 598/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:29.265332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f07605-9195-4304-adf5-75168d0fcae5 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:29.284205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:29.363837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f07605-9195-4304-adf5-75168d0fcae5 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:29.397014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:29.397786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.398064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.398334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.408033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:29.415684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:29.416819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:08:29.417984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 590/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:08:29.422697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:29.496194 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:29.518795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eec366-a3b9-4b38-85e4-a218217f74fe None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:29.550982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:29.551636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.551916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.552723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.553463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:29.553771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:29.555140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:29.562944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 599/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:08:29.569152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:29.646915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:29.667681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db14dcb-2fe1-4e8d-9604-8134643f09dd None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:29.700618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:29.701461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.701740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.702012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] PATH_INFO: `/v3/users/ef8e2e59563e46449a407413c05ebd84` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.703163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] RBAC: Authorizing `identity:get_user(user_id=ef8e2e59563e46449a407413c05ebd84)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:29.704637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:29.706218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 591/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:29 2022] GET /identity/v3/users/ef8e2e59563e46449a407413c05ebd84 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:29.711758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:29.806801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f67e7-4353-47df-a812-05343e2c90d4 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:29.838687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:29.839531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.839811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.840084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] PATH_INFO: `/v3/projects/8b735d8de1484445a6f71827eb0cc943` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.841265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] RBAC: Authorizing `identity:get_project(project_id=8b735d8de1484445a6f71827eb0cc943)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:29.842676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89d944-028d-4043-935d-246c7887fe5e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:29.844340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 600/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:29 2022] GET /identity/v3/projects/8b735d8de1484445a6f71827eb0cc943 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:29.850315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:29.946075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c455b-c898-4cd1-8f47-de2af8561b78 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:29.978092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:29.979203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:29.979479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:29.979744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] PATH_INFO: `/v3/projects/8b735d8de1484445a6f71827eb0cc943/users/ef8e2e59563e46449a407413c05ebd84/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:29.981684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] RBAC: Authorizing `identity:create_grant(project_id=8b735d8de1484445a6f71827eb0cc943, user_id=ef8e2e59563e46449a407413c05ebd84, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:29.983095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28071e2-dc69-4330-b41f-5e0d491a8dda None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:29.995730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 592/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:08:29 2022] PUT /identity/v3/projects/8b735d8de1484445a6f71827eb0cc943/users/ef8e2e59563e46449a407413c05ebd84/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:08:32.139280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb28d6-4fa7-4832-823d-68d295263aec None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.139796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb28d6-4fa7-4832-823d-68d295263aec None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.139897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb28d6-4fa7-4832-823d-68d295263aec None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.141211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 601/1193] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:08:32.146429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.146813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.146971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.158197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:32.164317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:08:32.166014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:08:32.230432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 593/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:08:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:08:32.253698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb28d6-4fa7-4832-823d-68d295263aec None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:32.263251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:32.352928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb28d6-4fa7-4832-823d-68d295263aec None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:32.385351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:32.386127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.386407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.386678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.397838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:08:32.405375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:08:32.406528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:08:32.407708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 602/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:08:32.412407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:32.507201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120617b0-431d-4c8b-a958-9197a6f77373 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:32.538988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:32.539633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.539909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.540175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.540936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:32.541234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:32.542605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:32.544702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:32.551096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 594/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:08:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:32.557060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:32.619449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:32.653308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96e872-bb95-448e-a20c-8d3f38e7cfb3 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:32.685651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:32.686502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.686781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.687052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] PATH_INFO: `/v3/users/ef8e2e59563e46449a407413c05ebd84` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.688215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] RBAC: Authorizing `identity:get_user(user_id=ef8e2e59563e46449a407413c05ebd84)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:32.689688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:32.691325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 603/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Tue May 24 15:08:32 2022] GET /identity/v3/users/ef8e2e59563e46449a407413c05ebd84 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:32.696850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:32.794343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdec6db-1369-4ced-bdc7-2d43a00c07d3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:32.826443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:32.827074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:32.827355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:32.827622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:32.888539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:32.892112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:32.893725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:32.987155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 595/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:08:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:08:32.993523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:33.088980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c538f-e09a-4cf3-bb39-612d7e9e84bc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:33.122409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:33.123257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:33.123532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:33.123801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] PATH_INFO: `/v3/projects/8b735d8de1484445a6f71827eb0cc943` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:33.124984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] RBAC: Authorizing `identity:get_project(project_id=8b735d8de1484445a6f71827eb0cc943)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:33.126411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e92924-0a90-48be-af14-3349c9632876 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:33.128067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 604/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Tue May 24 15:08:32 2022] GET /identity/v3/projects/8b735d8de1484445a6f71827eb0cc943 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:33.134053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:08:33.225448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:33.229720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5510bd71-d2c6-4312-9c1e-1f88a46147b0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:08:33.262988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:08:33.263630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:08:33.263910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:08:33.264183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:08:33.265037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=8b735d8de1484445a6f71827eb0cc943, user.id=ef8e2e59563e46449a407413c05ebd84) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:08:33.265347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:08:33.266794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd1ebf-b147-4fe4-b2b7-cf97a758e6b0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:08:33.272957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 596/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:08:33 2022] GET /identity/v3/role_assignments?user.id=ef8e2e59563e46449a407413c05ebd84&scope.project.id=8b735d8de1484445a6f71827eb0cc943&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:08:43.118438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:08:43.269901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=19 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:33.247372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8393-ce9c-44b5-867f-e8a82cceec27 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:33.247974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8393-ce9c-44b5-867f-e8a82cceec27 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:33.247974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8393-ce9c-44b5-867f-e8a82cceec27 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:33.249345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 605/1201] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:33.256333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:33.257328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:33.257638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:33.281369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:33.282652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:33.370893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 597/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:09:33.400216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df8393-ce9c-44b5-867f-e8a82cceec27 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:33.533517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df8393-ce9c-44b5-867f-e8a82cceec27 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:33.577200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:33.578002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:33.578363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:33.578640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:33.591480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:33.593177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:33.614000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:33.615016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:33.615976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:33.615976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:33.615976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:33.615976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:33.615976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 606/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:33.624026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:33.754282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91942b-93b7-4f53-9ad5-80cadba3f895 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:33.791182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:33.797895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:33.798650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:33.798981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:33.799145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:33.799853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:33.800157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:33.801581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbf0519-ed93-444d-8860-62bb9d7cf60e None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:33.813746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 598/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:33.820032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:33.885813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:33.945239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719e50f-94af-430b-b8ef-2f3597e1a278 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:33.989359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:33.989891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:33.990177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:33.990456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:33.991167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:33.992642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f122faf1-bce7-4cd7-b8c2-8dffadacb763 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:34.028613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 607/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Tue May 24 15:09:33 2022] POST /identity/v3/users => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:09:36.461102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47855fd-00fd-4537-a260-ce766b47714d None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:36.461628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47855fd-00fd-4537-a260-ce766b47714d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:36.461628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47855fd-00fd-4537-a260-ce766b47714d None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:36.463280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 599/1206] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:36.468735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:36.469230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:36.469351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:36.488754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:36.490515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:36.571259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 608/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:09:36.597576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47855fd-00fd-4537-a260-ce766b47714d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:36.717977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47855fd-00fd-4537-a260-ce766b47714d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:36.758870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:36.759705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:36.760111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:36.760409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:36.772905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:36.780805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:36.782289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:36.783352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 600/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:36.789307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:36.841980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:36.922808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6270ae-a80a-4c39-8396-40f713133fd2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:36.976777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:36.977387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:36.977666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:36.977936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:36.978954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:36.979257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:36.982464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:36.995705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 609/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:36.997092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:37.060932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:37.128338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af01771-1fc5-4834-b143-168abb268928 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:37.168203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:37.169121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:37.169401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:37.169666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:37.182046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:37.183683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:37.207443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:37.208596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:37.210059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:37.210059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:37.210059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:37.210059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:37.210059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 601/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:37.214762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:37.335803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:37.338538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e4dd8-5243-4667-a404-56763c05b323 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:37.381258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:37.382072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:37.382587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:37.382963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:37.383788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:37.384213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:37.385769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:37.400346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 610/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:37.405275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:37.520979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d783c-69d4-4c26-9082-2be8b78d5159 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:37.522746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:37.564117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:37.565051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:37.565422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:37.565724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:37.574356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:37.576229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:09:37.594358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:37.595352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:37.596352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 602/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:09:37.602849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:37.729288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa5af8-6562-4626-b35f-1fb070c33b98 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:37.774474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:37.775293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:37.775659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:37.775940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:37.776965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:37.777274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:37.778729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:37.801653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 611/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:09:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:37.807169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:37.931785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2064d4a-c748-41f8-8698-4fb23f8e6164 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:37.971428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:37.972280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:37.972684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:37.972960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:38.009600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:38.049418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:38.051566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:38.171683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 603/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:09:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:09:38.178104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:38.200022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:38.294649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16b2b-e889-42d0-9290-ec8593c76e6f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:38.332617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:38.333926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:38.334520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:38.335151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:38.349949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:38.352188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:38.375698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:38.376857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:38.377952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:38.377952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:38.377952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:38.377952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:38.379519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 612/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:09:38.385313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:38.450373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:38.495944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5633ae72-d57d-4879-a196-33ce4cfcd273 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:38.533812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:38.535249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:38.535673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:38.536089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:38.537509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:38.537863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:38.539487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:38.552170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 604/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:38.558626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:38.641590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:38.679902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff48e1-d4c3-43b9-86ff-69bfdfc382e5 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:38.723883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:38.724889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:38.725266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:38.725543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:38.734309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:38.736063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:09:38.751090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:38.752111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:38.753129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 613/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:38.758216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:38.867963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9adfb-7ac5-48ec-8acb-a325ba1633f9 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:38.891479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:38.904350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:38.905082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:38.905432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:38.905766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:38.906624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:38.906994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:38.908539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8db0533-e1c4-4b96-990c-5157bfac5c5a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:38.921754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 605/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:09:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:38.928391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:39.041195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8bb7-9e5a-4fb2-92d0-6e48bd1b6b6b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:39.075021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:39.083589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:39.084354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:39.084711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:39.085059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:39.086007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=1ac56ca4fe95468784f955de52375920) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:39.086396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:39.087823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1143f-f8cc-4239-a773-a3a94b1fcf54 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:39.094687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 614/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:09:38 2022] GET /identity/v3/role_assignments?user.id=1ac56ca4fe95468784f955de52375920&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:41.325227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f624-7215-41b3-90c7-f875393efbbe None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:41.325757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f624-7215-41b3-90c7-f875393efbbe None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:41.325853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1f624-7215-41b3-90c7-f875393efbbe None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:41.327563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 606/1220] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:41.333246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:41.333733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:41.333826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:41.351753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:41.353133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:41.421339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 615/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:09:41.445810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1f624-7215-41b3-90c7-f875393efbbe None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:41.560316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1f624-7215-41b3-90c7-f875393efbbe None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:41.597216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:41.597876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:41.598187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:41.598491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:41.609712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:41.618335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:41.619495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:41.620703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 607/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:09:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:41.625581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:41.725025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740bf54-119f-481e-9e4b-78d9daa8875b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:41.750214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:41.762655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:41.763575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:41.763856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:41.764128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:41.764918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] RBAC: Adding query filter params (name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:41.765227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:41.766631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:41.780086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 616/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:09:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:41.781538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:41.841006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:41.906076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7744fc-a325-42c6-a246-be064c4f3a9e None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:41.944098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:41.944913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:41.945190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:41.945460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:41.956636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:41.958134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:41.976042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:41.977309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:41.979216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:41.979216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:41.979216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:41.979216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:41.979216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 608/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:41.984739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.095681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1fd290-ee16-4b40-901a-b51398ba7c27 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:42.139462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:42.140135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:42.140454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:42.140744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:42.141473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:42.141753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:42.143144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:42.155608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 617/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:42.162034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.263509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:42.270956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d455e4-37a9-4002-9e62-dffda2ad6ad7 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:42.307890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:42.308944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:42.309273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:42.309592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:42.317992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:42.319452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:09:42.337394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:42.338970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:42.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 609/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:42.345679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.397015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:42.454593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6fd6be-086e-459f-b57e-e5c32b1857e2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:42.493936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:42.494665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:42.494950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:42.495227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:42.496088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:42.496437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:42.497905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:42.523494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 618/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:09:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:42.529789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.639485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f183c3-4138-42e2-a2c8-59c03b6a99a8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:42.676968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:42.678950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:42.679290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:42.679666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:42.691868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:42.693581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:42.710398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:42.711519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:42.712486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:42.712486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:42.712486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:42.712486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:42.712486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 610/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:42.717461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.827421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba763b-456b-457b-a652-0aa2dc369a43 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:42.830220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:42.868807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:42.869558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:42.869847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:42.870186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:42.870996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:42.871333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:42.873018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:42.886801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 619/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:42.892940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:42.902390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:43.004613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7a2bb-3cc2-49d0-b3b5-9dfe382775fe None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:43.040415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:43.041209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:43.041486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:43.041761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:43.048479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:43.049953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:09:43.063968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:43.064981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:43.066025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 611/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:43.071523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:43.180156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e155fa6-8c71-4924-9e43-76981f369a2d None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:43.215805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:43.216506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:43.216789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:43.217062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:43.217969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:43.218366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:43.220110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80bcb7-af8a-4479-9127-05858256b23b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:43.233418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 620/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:09:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:43.239577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:43.309297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:43.349833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2fecf-a291-43c3-8739-b8603d730052 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:43.388361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:43.389560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:43.389848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:43.390125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/1ac56ca4fe95468784f955de52375920/roles/6dfdc8d7453540e9a62421b124880570` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:43.408240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=1ac56ca4fe95468784f955de52375920, role_id=6dfdc8d7453540e9a62421b124880570)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:43.410383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd90c9-c95a-4a79-9b9b-be2443b8d1fa None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:43.423852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 612/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:09:43 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/1ac56ca4fe95468784f955de52375920/roles/6dfdc8d7453540e9a62421b124880570 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:09:45.774990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f157-375e-48ef-bab0-29a386396949 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:45.775568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f157-375e-48ef-bab0-29a386396949 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:45.775568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5f157-375e-48ef-bab0-29a386396949 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:45.777100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 621/1233] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:45.782284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:45.782799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:45.782893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:45.800739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:45.802284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:45.883379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 613/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:09:45.906944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f157-375e-48ef-bab0-29a386396949 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:45.934895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:46.019641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5f157-375e-48ef-bab0-29a386396949 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.056498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.057408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.057683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.057954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.069229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:46.076919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:46.077956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 15:09:46.078810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:46.083605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 622/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Tue May 24 15:09:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:46.084700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:46.131870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:46.193343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e863127-cbad-49c8-8009-0e1bd6383325 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.232213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.232920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.233199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.233471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.234228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] RBAC: Adding query filter params (name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:46.234536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:46.235997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:46.245802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 614/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:46.251521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:46.348813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:46.368120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d004d1-5a51-4cae-902f-5c24df87d27f None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.405669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.406565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.406849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.407212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.419677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:46.421312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:46.439227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:46.440827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:46.441741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:46.441741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:46.441741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:46.441741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:46.441741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 623/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:46.447855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:46.554327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e89829-f120-499b-a3f0-2d819cc9e830 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.563997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:46.593411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.594051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.594333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.594675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.595347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:46.595619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:46.597122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:46.608870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 615/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:46.615033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:46.721921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b675a-665f-42d5-99b2-03b58694ef75 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.759844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.760692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.760994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.761319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.764397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:46.771124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:46.772947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:09:46.790089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:46.791527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:46.792506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 624/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:46.799532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:46.924316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a84565-4ff0-462b-b968-9622d6d4367f None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:46.959719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:46.960550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:46.960885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:46.961147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:46.962059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:46.962383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:46.963940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:46.984483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 616/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:46.990472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:47.100747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a54017-7101-4eed-a20f-b20b6297bb17 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:47.139096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:47.139746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:47.140026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:47.140317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:47.205528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:47.213745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:47.215225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:47.319343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 625/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:09:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:09:47.325158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:47.355584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:47.434045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0545e88-f5dd-46c9-8c34-f74dc4bcd77d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:47.478895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:47.479739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:47.480075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:47.480373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:47.491040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:47.492534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:47.511109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:47.512485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:47.513558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:47.513558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:47.513558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:47.513558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:47.513558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 617/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:47.518601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:47.624226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:47.630893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14334a24-a787-4910-8471-a1ce20ca98ae None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:47.666652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:47.667317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:47.667620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:47.667903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:47.668637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:47.668904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:47.670266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:47.682198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 626/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:47.687904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:47.776137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:47.799136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c4591-d1f2-4486-8577-db7f0cc83fdb None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:47.839556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:47.840662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:47.841058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:47.841385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:47.850785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:47.852419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:09:47.868362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:47.869507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:47.870615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 618/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:47.876254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:48.007727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bc431-dcf5-4e24-b830-a2a55664cd78 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:48.041840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:48.042428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:48.042709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:48.042980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:48.043762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:48.044071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:48.045557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb2469-a25a-4adb-8588-f479fed48cdb None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:48.058247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 627/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:48.064441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:48.191657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e218527-47c4-4f7e-88b1-971a5255335a None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:48.231612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:48.232457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:48.232815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:48.233129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:48.234067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] RBAC: Adding query filter params (role.id=6dfdc8d7453540e9a62421b124880570, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=1ac56ca4fe95468784f955de52375920) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:48.234407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:48.235858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e7ed7-d6cd-408b-8023-ccf94c1b4678 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:48.242515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 619/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:09:48 2022] GET /identity/v3/role_assignments?user.id=1ac56ca4fe95468784f955de52375920&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfdc8d7453540e9a62421b124880570 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:50.614016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1b75d-1109-4c54-b2d5-330143f6e06a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:50.614560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1b75d-1109-4c54-b2d5-330143f6e06a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:50.614676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1b75d-1109-4c54-b2d5-330143f6e06a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:50.616027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 628/1247] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:50.621424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:50.621861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:50.622038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:50.627061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:50.641102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:50.642466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:50.717203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 620/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:09:50.744278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1b75d-1109-4c54-b2d5-330143f6e06a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:50.763279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:50.867637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1b75d-1109-4c54-b2d5-330143f6e06a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:50.909415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:50.910202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:50.910482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:50.910786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:50.921559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:50.929603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:50.930587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:50.931458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 629/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:09:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:50.936938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.046632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fc637-c256-4268-a9d6-43731ebe89ef None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.081578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:51.086966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.087872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.088155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.088466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:51.089227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:51.089553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:51.090952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:51.103792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 621/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:09:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:51.109503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.200458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:51.225648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235ddd-e9a0-493d-801f-df8b234ba0e0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.261125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.261967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.262288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.262609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:51.273814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:51.275272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:51.292728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:51.294042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:51.296083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:51.296083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:51.296083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:51.296083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:51.296083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 630/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:09:51.300338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.411270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965467b9-e19b-4a0c-8f71-1ccad7c0cb73 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.446379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.447036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.447609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.447810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:51.448553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:51.448863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:51.450236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:51.462376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 622/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:51.469057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.578878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba4496-edb2-4c62-8dbf-f6a2b1366fff None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.609225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:51.618214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.619255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.619633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.619916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:51.629471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:51.631116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:09:51.646692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:51.647814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:51.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 631/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:51.654404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.717755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:51.766054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384b7ad-f373-4f85-90de-a384f4766688 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.804646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.805326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.805633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.805907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:51.806743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:51.807056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:51.808617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:51.831921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 623/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:09:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:51.837792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:51.948946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae9938-f7f4-44c9-90b3-295c9b67d929 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:51.985810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:51.986378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:51.987186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:51.987447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.059223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:52.069049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.070541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.173618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 632/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:09:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 15:09:52.180211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:52.283671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1121c-c325-4d83-820f-6b55f25eafb8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:52.297311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:52.320090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:52.320930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:52.321209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:52.321478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.332814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.334305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:52.353238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:52.354510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:52.355600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:52.355600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:52.355600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:52.355600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:52.355600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 624/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:52.360727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:52.451370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:52.463818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e8737-6108-4e8c-acec-97820e19b0b2 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:52.499930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:52.500655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:52.500939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:52.501213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.501937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:52.502299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.503684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.515068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 633/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:52.520847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:52.623701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17048b-b9d9-4356-a813-b9c1f741d809 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:52.656658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:52.657451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:52.657731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:52.658003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.665375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.666836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:09:52.680721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:52.681625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:52.682824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 625/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:52.687944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:52.787192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84839f6-f045-43c5-8be4-1a053a7760fc None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:52.821614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:52.822359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:52.822688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:52.823021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.824017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:52.824230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.825825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227345e7-b465-412e-a39a-bb83686cd053 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.839595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 634/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:09:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:52.845799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:52.873153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:52.944578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8075e-1740-4c39-9f32-4c6912a14db3 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:52.976960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:52.977586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:52.977860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:52.978127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:52.979212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=1ac56ca4fe95468784f955de52375920) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:52.979517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:52.980987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc7ff8-5494-4304-ba98-ab571d6556e4 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:52.986183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 626/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:09:52 2022] GET /identity/v3/role_assignments?user.id=1ac56ca4fe95468784f955de52375920&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:55.112698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802956-baaf-47f4-a3f4-b5cbc9793347 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.113192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802956-baaf-47f4-a3f4-b5cbc9793347 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.113290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07802956-baaf-47f4-a3f4-b5cbc9793347 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.114587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 635/1261] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:55.119647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.120035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.120194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.137216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:55.138379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:55.203089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 627/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:09:55.225091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07802956-baaf-47f4-a3f4-b5cbc9793347 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:55.235969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:55.323536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07802956-baaf-47f4-a3f4-b5cbc9793347 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:55.356029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:55.356827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.357105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.357375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.367818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:55.375322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:55.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:55.377693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 636/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:09:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:55.382518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:55.400062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:55.477237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca5a6c-d6b4-4898-a33a-a76ea85ade7e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:55.514758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:55.515413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.515689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.515957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.516723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:55.517025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:55.518390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:55.526069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 628/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:09:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:55.531938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:55.594818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:55.627306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf9d3-d4bf-4f72-8f4e-c2abcc7a1c82 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:55.658876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:55.660241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.660545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.660814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.670863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:55.672345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:55.688707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:55.689739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:55.695472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:55.695472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:55.695472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:55.695472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:55.695472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 637/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:55.695846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:55.773957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:55.793474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb81111-2612-4250-a690-a1011d23f5ea None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:55.829900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:55.830794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.831649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.832092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.832973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:55.833418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:55.835076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:55.850311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 629/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:55.856639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:55.953190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbbe12-14b7-47f0-b9e7-5a131148d77b None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:55.972943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:55.988900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:55.989694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:55.989971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:55.990243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:55.997572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:55.998989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:09:56.012731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:56.013850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:56.014949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 638/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 15:09:56.020086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.115747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6e48-d62c-4061-86bd-526ba2b18463 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.148712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.149365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.149641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.149911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.150727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:56.151030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.152477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.160815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:56.169295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 630/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:09:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:56.174831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.272542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c658a2-0df1-4fc8-90c1-3c7cd9486e30 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.305849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.306653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.306935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.307208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.317848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.319293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:56.336384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:56.337411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:56.338514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:56.338514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:56.338514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:56.338514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:56.338514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 639/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:56.343189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.439645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12999927-5dfd-4d7a-acb7-85c2779cf4d6 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.472416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.473062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.473343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.473615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.474329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:56.474633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.476000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.486990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 631/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:56.492892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.523693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:56.591233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bacde7-6ee5-4a78-92bf-2f99b421d973 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.623693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.624509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.624786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.625050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.632093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.633554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:09:56.647214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:56.648384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:09:56.649300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 640/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:56.654191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.710307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:56.750229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb884f6-5609-4536-8610-09632a409685 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.782054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.782691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.782967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.783238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.784008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:56.784857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.786284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9cf64-7705-425b-a87f-9bc8c27913af None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.797719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 632/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:09:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:56.804046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:56.894710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:56.903365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669f890-8abf-4ba8-9b5b-5e0b7778baa1 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:56.936376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:56.937243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:56.937524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:56.937796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] PATH_INFO: `/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/1ac56ca4fe95468784f955de52375920/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:56.939706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=c6da052fcc124bd5b71972a9a0e094fa, user_id=1ac56ca4fe95468784f955de52375920, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:56.941178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ace7-a85f-440d-8a48-2c88b6552bdf None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:56.952775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 641/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Tue May 24 15:09:56 2022] PUT /identity/v3/projects/c6da052fcc124bd5b71972a9a0e094fa/users/1ac56ca4fe95468784f955de52375920/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:09:59.132909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36641c8-9aef-440f-a8cf-28b420185581 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.133392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36641c8-9aef-440f-a8cf-28b420185581 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.133502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36641c8-9aef-440f-a8cf-28b420185581 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.134808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 633/1274] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:09:59.139883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.140262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.140483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.160617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:09:59.161818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:09:59.226021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 642/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:09:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:09:59.249459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36641c8-9aef-440f-a8cf-28b420185581 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:59.346890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36641c8-9aef-440f-a8cf-28b420185581 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:59.363376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:59.379964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:59.380769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.381045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.381312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.390936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 15:09:59.399054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:59.400133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:59.401260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 634/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Tue May 24 15:09:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:59.406196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:59.485150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:59.504526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1a5b3-1c7d-4ced-aa42-dd9a32d188e6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:59.536994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:59.537654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.537936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.538208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.538948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:59.539253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:59.540677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:59.548428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 643/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Tue May 24 15:09:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:09:59.554609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:59.651355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb003c4a-a14b-43e7-82da-7cdb61c4a3ad None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:59.683645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:59.684433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.684716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.684991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.695711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:59.697199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:09:59.713606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:09:59.714634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:09:59.715861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:09:59.715861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:09:59.715861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:09:59.715861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:09:59.715861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 635/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:09:59.720210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:59.824549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caed201f-1025-4e40-8d2b-7da3875b8cdd None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:09:59.866648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:09:59.867316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:09:59.867596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:09:59.867865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:09:59.868692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:09:59.869034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:09:59.870395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:09:59.882043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 644/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:09:59.887649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:09:59.917729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:09:59.985391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98d94d-3c52-4b83-8e74-fc27e19b6f34 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.017440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.018214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.018584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.019354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.027193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.028673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 15:10:00.042770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:00.043868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:00.045090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 636/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:09:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:00.049581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:00.053970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:00.147203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1f769-96e4-41ce-b755-322ec5d290c0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.179856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.180487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.180764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.181032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.181848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:00.182149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.183557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.199732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 645/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Tue May 24 15:10:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:00.205388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:00.297296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:00.307186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5690b4e-8c1e-4492-8457-b4f510b66a96 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.339208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.339835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.340111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.340416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.404136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.405607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.491236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:00.501738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 637/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Tue May 24 15:10:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 15:10:00.508051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:00.566924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:00.603926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae65fc7-8846-49e3-a133-cfd0c2bae4a7 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.636239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.637047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.637327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.637599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.647271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.648738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 15:10:00.666034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:00.667079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:00.668049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 15:10:00.668049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 15:10:00.668049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 15:10:00.668049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:10:00.668049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 646/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:00.672778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:00.776767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736e083-a86b-40ee-8f99-22f77bf04b57 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.823899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.824615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.824849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.825156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.825900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:00.826139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.827529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.840105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 638/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Tue May 24 15:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:00.845685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:00.941633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36adba2d-1b4a-45ca-8934-45d6aa12255a None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:00.954869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:00.975218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:00.975997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:00.976277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:00.976582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:00.983307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:00.984776 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 15:10:00.998118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application with Lock( May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:00.999140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: ERROR keystone.server.flask.application May 24 15:10:01.000495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 647/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Tue May 24 15:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:01.005148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:01.054732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:01.108500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a325b31-f125-42d8-b848-bd839893cfb5 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:01.140597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:01.141250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:01.141533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:01.141806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:01.142589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:01.142901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:01.144912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ddcc9e-4422-4056-89d7-79213aa57f57 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:01.156458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 639/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Tue May 24 15:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:01.163168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:01.258691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058e2c84-3128-4c1e-8bfe-bd76bad500f0 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:01.291411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:01.292044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:01.292340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:01.292614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:01.293435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] RBAC: Adding query filter params (role.id=6dfe90fc0a2441a881142526cd3c7974, scope.project.id=c6da052fcc124bd5b71972a9a0e094fa, user.id=1ac56ca4fe95468784f955de52375920) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:01.293747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:01.295173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b7d0b2-9a13-4f5e-8260-6c03580209ef None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:01.300877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 648/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Tue May 24 15:10:01 2022] GET /identity/v3/role_assignments?user.id=1ac56ca4fe95468784f955de52375920&scope.project.id=c6da052fcc124bd5b71972a9a0e094fa&role.id=6dfe90fc0a2441a881142526cd3c7974 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:03.490063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067272-ab21-47f3-a749-7b46a984e80e None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:03.490599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067272-ab21-47f3-a749-7b46a984e80e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:03.490599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07067272-ab21-47f3-a749-7b46a984e80e None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:03.491970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 640/1288] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:10:03.497266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:03.497686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:03.497818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:03.514924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:10:03.516090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:10:03.545265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:03.581973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 649/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:10:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:10:03.604496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067272-ab21-47f3-a749-7b46a984e80e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:03.702117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07067272-ab21-47f3-a749-7b46a984e80e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:03.713632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:03.736213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:03.737023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:03.737331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:03.737607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:03.738250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:03.739638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:03.751153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:03.752363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:03.753803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 641/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:10:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:03.758629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:03.860733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede16d4-27bf-4e05-816b-ff641d70821e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:03.895060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:03.895724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:03.896009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:03.896281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:03.897009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:03.897311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:03.898682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb5e14-bddd-4af2-aca6-c76c8b5e3c83 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:03.903405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 650/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 15:10:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:03.909120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:04.005490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89589088-558c-4b0f-86f6-7a358ed079b0 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:04.036836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:04.037481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:04.037762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:04.038036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] PATH_INFO: `/v3/services` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:04.038730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] RBAC: Adding query filter params (type=placement) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:04.039034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:04.040426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73e820-640e-425e-8483-d79b1b1ac0be None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:04.044718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 642/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 15:10:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:06.189403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e6e5-cae2-4212-9fed-4031f6ac7b4a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:06.189905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e6e5-cae2-4212-9fed-4031f6ac7b4a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:06.189996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03e6e5-cae2-4212-9fed-4031f6ac7b4a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:06.191879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 651/1293] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:10:06.196947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcc06-157d-4f9f-a6d4-6be0217ad9ce None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:06.197384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcc06-157d-4f9f-a6d4-6be0217ad9ce None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:06.197487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dcc06-157d-4f9f-a6d4-6be0217ad9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:06.213806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9f5dcc06-157d-4f9f-a6d4-6be0217ad9ce None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:10:06.214984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9f5dcc06-157d-4f9f-a6d4-6be0217ad9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:10:06.243967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:06.280180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 643/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:10:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 15:10:06.302368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03e6e5-cae2-4212-9fed-4031f6ac7b4a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:06.332548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:06.400087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03e6e5-cae2-4212-9fed-4031f6ac7b4a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:06.432528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:06.433176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:06.433454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:06.433723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:06.434433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:06.435781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660257aa-d488-4e24-841a-ad58feee85d0 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:06.448666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 652/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Tue May 24 15:10:06 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:10:08.595834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888c9e-0cfc-44fa-9e4b-9e8a036cb0e0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:08.596398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888c9e-0cfc-44fa-9e4b-9e8a036cb0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:08.596495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a888c9e-0cfc-44fa-9e4b-9e8a036cb0e0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:08.597846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 644/1296] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:10:08.602855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:08.603222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:08.603659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:08.620343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:10:08.621507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:10:08.689780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 653/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:10:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:10:08.711883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a888c9e-0cfc-44fa-9e4b-9e8a036cb0e0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:08.810071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a888c9e-0cfc-44fa-9e4b-9e8a036cb0e0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:08.846049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:08.847286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:08.847566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:08.847883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:08.848516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:08.849902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:08.863792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:08.865007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:08.865965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 645/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:10:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:08.871734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:08.893153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:08.970934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f551d-82cd-4e6f-a84c-d04433fb3452 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:09.004338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:09.005000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:09.005282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:09.005557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:09.006255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:09.006561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:09.007924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:09.012955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 654/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 15:10:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:09.018780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:09.049417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:09.118027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b57267-95a7-4267-999d-06953414bfe8 None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:09.149935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:09.150580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:09.150861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:09.151136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:09.151914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] RBAC: Adding query filter params (interface=public, service_id=a54d0f5d9ebf4a1ab8c0b0763b6d5765, region_id=RegionOne) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:09.152225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:09.153621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a4f0e-4af6-4c0e-806b-fcea7f3e083d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:09.158507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 646/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Tue May 24 15:10:09 2022] GET /identity/v3/endpoints?service_id=a54d0f5d9ebf4a1ab8c0b0763b6d5765&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:10:09.164648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:09.242319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:09.259848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288158ae-b03a-4e37-a18d-9dc666e9e267 None admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:09.292849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:09.293495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:09.293773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:09.294044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:09.294707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:09.296055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca43eca8-73a6-4107-9ea2-1c576400ee73 None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:09.302671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 655/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Tue May 24 15:10:09 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:10:11.430288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7e2a-b5a4-4ed6-a2b1-9e1fb2fbbed7 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:11.430807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7e2a-b5a4-4ed6-a2b1-9e1fb2fbbed7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:11.430807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd7e2a-b5a4-4ed6-a2b1-9e1fb2fbbed7 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:11.432145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 647/1302] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:10:11.437247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:11.437668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:11.437778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:11.455325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:10:11.456530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:10:11.520979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 656/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:10:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 15:10:11.543110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd7e2a-b5a4-4ed6-a2b1-9e1fb2fbbed7 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:11.636342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:11.643373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd7e2a-b5a4-4ed6-a2b1-9e1fb2fbbed7 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:11.675464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:11.676243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:11.676549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:11.676819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:11.677462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:11.678833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 15:10:11.691042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 15:10:11.692225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.get_or_create( May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application with Lock( May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self._enter() May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application return self.creator() May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application created_value = creator( May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: ERROR keystone.server.flask.application May 24 15:10:11.693423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 648/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Tue May 24 15:10:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 15:10:11.698289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:11.771040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:11.794245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5c61f-2a84-4604-acee-9036f9142773 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:11.835219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:11.835882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:11.836161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:11.836468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] PATH_INFO: `/v3/services` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:11.837175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:10:11.837479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:11.838846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af21ccf-5812-4f81-8bd7-b910643a055e None admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:11.843897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 657/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Tue May 24 15:10:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:10:11.850151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:10:11.954603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9381e-55da-4a16-b687-4552d6c8607d None admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:10:11.987808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:10:11.988485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:10:11.988763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:10:11.989031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:10:11.989774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:10:11.991154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a72e87-3f43-4bcb-afc4-a25febf1e450 None admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:10:12.008375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 649/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Tue May 24 15:10:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:10:21.840975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=30 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:10:22.004545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:15.107163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdc0a6-9d83-4b26-b973-dae2d7618546 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:15.107741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdc0a6-9d83-4b26-b973-dae2d7618546 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:15.107985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdc0a6-9d83-4b26-b973-dae2d7618546 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:15.109696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 658/1307] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:15.115585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8862-7a47-4374-a013-4dd65ca58203 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:15.116501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8862-7a47-4374-a013-4dd65ca58203 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:15.116953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a8862-7a47-4374-a013-4dd65ca58203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:15.264190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-386a8862-7a47-4374-a013-4dd65ca58203 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:15.265787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-386a8862-7a47-4374-a013-4dd65ca58203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:15.404887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 650/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:15.451261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:15.451938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:15.452415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:15.454439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 659/1309] 10.222.0.45 () {54 vars in 964 bytes} [Tue May 24 15:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:15.463916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50565f81-af2f-4591-a04e-6416f131859a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:15.464364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50565f81-af2f-4591-a04e-6416f131859a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:15.464745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50565f81-af2f-4591-a04e-6416f131859a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:15.559111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-50565f81-af2f-4591-a04e-6416f131859a None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:15.560515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-50565f81-af2f-4591-a04e-6416f131859a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:15.582067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:15.641898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 651/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Tue May 24 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:15.651630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:15.653495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:15.773061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc7907-c3ce-4ed5-8136-3ef891173211 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:15.852605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:15.853440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:15.853815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:15.854180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:15.856216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:15.912650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:15.973980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:15.975514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eee86-3276-4b30-8f71-00342a8380c0 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:15.996469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 660/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:18.520276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:18.520831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:18.520948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:18.522231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 652/1312] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:18.527401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d913-d940-4ea2-a145-6f95cc5c124e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:18.527787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d913-d940-4ea2-a145-6f95cc5c124e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:18.527960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252d913-d940-4ea2-a145-6f95cc5c124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:18.608217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c252d913-d940-4ea2-a145-6f95cc5c124e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:18.609454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c252d913-d940-4ea2-a145-6f95cc5c124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:18.704136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 661/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:18.742094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:18.743626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:18.845205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fecf2-3f29-4e0f-a14d-946df80d3e92 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:18.918168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:18.920982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:18.921854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:18.922142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:18.922604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:18.924587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:19.030526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:19.031973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8d7508-e4a4-44dd-9cf3-7183d2904c05 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:19.050139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 653/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:21.566953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:21.567475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:21.567475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:21.568907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 662/1315] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:21.574018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21ea27-9027-43de-a079-62a12a491159 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:21.574442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21ea27-9027-43de-a079-62a12a491159 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:21.575588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21ea27-9027-43de-a079-62a12a491159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:21.638793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bc21ea27-9027-43de-a079-62a12a491159 None None] MFA Rules not processed for user `42b1dfbd9a6b4e23b6dc61a02d6ea243`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:21.639981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bc21ea27-9027-43de-a079-62a12a491159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:21.690967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:21.736529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 654/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:21.776131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:21.777718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:21.788163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:21.882931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0a12d-d61a-47cc-aef7-627330ef36d9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:21.958698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:21.959353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:21.959634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:21.959930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:21.961577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:22.023371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:22.070881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:22.072357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1c04f-00ca-4671-b961-1c91c55be957 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:22.091278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 663/1317] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:24.839228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26789b84-9b75-4733-99ee-ce473e6dfbc0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:24.839672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26789b84-9b75-4733-99ee-ce473e6dfbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:24.839763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26789b84-9b75-4733-99ee-ce473e6dfbc0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:24.841185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 655/1318] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:24.846259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc37b-7181-4ac9-913c-4374639ecf49 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:24.846691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc37b-7181-4ac9-913c-4374639ecf49 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:24.846812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fc37b-7181-4ac9-913c-4374639ecf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:24.912020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-961fc37b-7181-4ac9-913c-4374639ecf49 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:24.913229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-961fc37b-7181-4ac9-913c-4374639ecf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:25.007293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 664/1319] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:25.060591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:25.060903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:25.061179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:25.062556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 656/1320] 10.222.0.45 () {54 vars in 964 bytes} [Tue May 24 15:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:25.068495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918a651-65dd-475c-a07f-dd9a5a1c4920 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:25.068785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918a651-65dd-475c-a07f-dd9a5a1c4920 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:25.069063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918a651-65dd-475c-a07f-dd9a5a1c4920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:25.090087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:25.142407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0918a651-65dd-475c-a07f-dd9a5a1c4920 None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:25.144078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0918a651-65dd-475c-a07f-dd9a5a1c4920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:25.216810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 665/1321] 10.222.0.45 () {58 vars in 1061 bytes} [Tue May 24 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:11:25.226347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:25.227862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:25.328605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1eab8-47ff-4988-9b7a-86eeb57e0a70 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:25.401062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:25.401638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:25.401922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:25.402199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:25.403832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:25.411201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:25.514228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:25.515674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd9f40-6fa2-40c8-82d7-b34db7ffedb0 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:25.533796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 657/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:28.829360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:28.829799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:28.829889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:28.831926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 666/1323] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:28.836985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af75b-360a-471a-8f29-4df3dec4977f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:28.837411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af75b-360a-471a-8f29-4df3dec4977f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:28.837596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59af75b-360a-471a-8f29-4df3dec4977f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:28.901489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e59af75b-360a-471a-8f29-4df3dec4977f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:28.902676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e59af75b-360a-471a-8f29-4df3dec4977f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:28.960194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:28.999777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 658/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:29.036167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:29.037734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:29.143645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b7981-1d92-419c-aa74-e295724932ae None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:29.155597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:29.216353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:29.217007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:29.217287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:29.217560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:29.219179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:29.327459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:29.329120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a72d560-d8d8-4e71-b38e-b4fc100d8f9e service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:29.348769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 667/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:31.636504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:31.637055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:31.637055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:31.638410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 659/1326] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:31.644230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c5c1-e022-46cd-9ceb-f4c394529143 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:31.644683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c5c1-e022-46cd-9ceb-f4c394529143 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:31.644944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74c5c1-e022-46cd-9ceb-f4c394529143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:31.690637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:31.709391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-cd74c5c1-e022-46cd-9ceb-f4c394529143 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:31.710576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-cd74c5c1-e022-46cd-9ceb-f4c394529143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:31.804419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 668/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:31.839655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:31.841207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:31.943045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc83fa-84ee-4d44-8611-9f3df541ea93 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:32.014504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:32.015159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:32.015421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:32.015695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:32.017319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:32.033902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:32.125285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:32.126765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3198d2-738c-47a9-b090-605dbd0f1903 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:32.145359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 660/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:11:34.419053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:34.419572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:34.419572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:34.420934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 669/1329] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:34.426337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cbfc-dba8-4a85-bef0-94c64fa5ddcc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:34.426757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cbfc-dba8-4a85-bef0-94c64fa5ddcc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:34.426928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4cbfc-dba8-4a85-bef0-94c64fa5ddcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:34.493047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-32e4cbfc-dba8-4a85-bef0-94c64fa5ddcc None None] MFA Rules not processed for user `42b1dfbd9a6b4e23b6dc61a02d6ea243`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:34.494237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-32e4cbfc-dba8-4a85-bef0-94c64fa5ddcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:34.559999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:34.593984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 661/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:34 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:34.631366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:34.633009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:34.735013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a937-8a92-4141-b33b-171222e6fe46 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:34.754577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:34.807832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:34.808508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:34.808786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:34.809059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:34.810668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:34.916566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:34.918000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6377693-8a75-48ed-b4ed-586f7b0e9d9f service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:34.935865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 670/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:34 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:37.383836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:37.384389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:37.384389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:37.385706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 662/1332] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:11:37.390899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdeb50-21ae-4cca-8cd8-40e60be38246 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:37.391303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdeb50-21ae-4cca-8cd8-40e60be38246 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:37.391461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdeb50-21ae-4cca-8cd8-40e60be38246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:37.454959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:37.458292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a7cdeb50-21ae-4cca-8cd8-40e60be38246 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:37.460043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a7cdeb50-21ae-4cca-8cd8-40e60be38246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:37.556003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 671/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:37.592242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:37.593787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:37.696792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea391c6-fc08-4167-a9ef-e754cf9946fb None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:37.770486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:37.771036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:37.771327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:37.771578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:37.773270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:37.802400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:37.890244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:37.891704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81727a29-1207-4889-9f78-7b49254d8d26 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:37.911454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 663/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:40.162471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:40.162976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:40.163089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:40.164432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 672/1335] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:40.169538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca716a88-918c-4dc3-950c-859065a1abb9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:40.169962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca716a88-918c-4dc3-950c-859065a1abb9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:40.170093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca716a88-918c-4dc3-950c-859065a1abb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:40.233671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ca716a88-918c-4dc3-950c-859065a1abb9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:40.234871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ca716a88-918c-4dc3-950c-859065a1abb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:40.308796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:40.332380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 664/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:40.371483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:40.373058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:40.475077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc01acb-af5e-4a6c-b447-d169aeb33052 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:40.503271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:40.547186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:40.547857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:40.548141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:40.548467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:40.550072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:40.654230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:40.655650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574cd01-f208-4a63-bef9-36c7d8638a5d service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:40.672904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 673/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:43.333589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:43.334148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:43.334148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:43.335492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 665/1338] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:43.340655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f26f-afde-4ab1-85aa-db655f32e9e8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:43.341040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f26f-afde-4ab1-85aa-db655f32e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:43.341215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f26f-afde-4ab1-85aa-db655f32e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:43.404563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7c99f26f-afde-4ab1-85aa-db655f32e9e8 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:43.405753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7c99f26f-afde-4ab1-85aa-db655f32e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:43.413872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:43.500802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 674/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:43.536756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:43.538312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:43.639014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837f76d-0a33-4715-8010-4833894a4a83 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:43.709346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:43.709981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:43.710256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:43.710521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:43.712098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:43.746554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:43.817881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:43.819326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b549c-1bce-4c2e-96c3-ce6c93f921e9 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:43.837273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 666/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:48.142342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:48.143410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:48.143410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:48.144789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 675/1341] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:48.150121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783579-d842-4e2a-ac99-af6e4b76e619 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:48.150541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783579-d842-4e2a-ac99-af6e4b76e619 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:48.150810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99783579-d842-4e2a-ac99-af6e4b76e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:48.219922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-99783579-d842-4e2a-ac99-af6e4b76e619 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:48.221727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-99783579-d842-4e2a-ac99-af6e4b76e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:48.316408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:48.330077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 667/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:48.369154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:48.371256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:48.467585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e969d27-0074-45eb-a214-4d0b926e893b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:48.471754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:48.472440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:48.472725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:48.472999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:48.482307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:48.483749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78f003e-03b4-4431-bef2-6eecd4897c1b service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:48.493488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 676/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:51.306492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8084aad-9c03-4c11-8df5-da240f1b9e8f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:51.307021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8084aad-9c03-4c11-8df5-da240f1b9e8f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:51.307021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8084aad-9c03-4c11-8df5-da240f1b9e8f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:51.308360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 668/1344] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:51.313454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259cd7-6aed-44c7-9905-2292b720cbb0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:51.313898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259cd7-6aed-44c7-9905-2292b720cbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:51.313997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c259cd7-6aed-44c7-9905-2292b720cbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:51.341446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0c259cd7-6aed-44c7-9905-2292b720cbb0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:51.342639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0c259cd7-6aed-44c7-9905-2292b720cbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:51.370635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:11:51.418746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 677/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:51.455982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8084aad-9c03-4c11-8df5-da240f1b9e8f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:51.466546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8084aad-9c03-4c11-8df5-da240f1b9e8f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:51.470026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:51.470662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:51.470940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:51.471212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:51.480266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:51.481737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e4a5b-8a62-4157-b590-8a6854439af0 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:51.491823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 669/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:11:55.558981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc1b6-4403-4b50-940f-6e8576d8dd67 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:55.559524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc1b6-4403-4b50-940f-6e8576d8dd67 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:55.559524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dc1b6-4403-4b50-940f-6e8576d8dd67 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:55.561394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 678/1347] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:11:55.566814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c18b-7335-49ce-90e8-674f812c4b59 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:55.567217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c18b-7335-49ce-90e8-674f812c4b59 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:55.567368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481c18b-7335-49ce-90e8-674f812c4b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:55.586625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2481c18b-7335-49ce-90e8-674f812c4b59 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:11:55.587935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2481c18b-7335-49ce-90e8-674f812c4b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:11:55.595749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 670/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:11:55.633628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dc1b6-4403-4b50-940f-6e8576d8dd67 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:11:55.645056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dc1b6-4403-4b50-940f-6e8576d8dd67 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:11:55.648618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:11:55.649273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:11:55.649555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:11:55.649829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:11:55.659394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:11:55.660860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf0f86-199f-4231-ba55-035f1d460402 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:11:55.670238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 679/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:01.051508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e61-938c-40e3-beb2-30375f19b3fc None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:01.052730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e61-938c-40e3-beb2-30375f19b3fc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:01.053225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e61-938c-40e3-beb2-30375f19b3fc None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:01.055149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 671/1350] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:12:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:01.059838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f757-c66d-4095-873c-3dab2e20b02a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:01.060724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f757-c66d-4095-873c-3dab2e20b02a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:01.061237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f757-c66d-4095-873c-3dab2e20b02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:01.168034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c685f757-c66d-4095-873c-3dab2e20b02a None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:01.169465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c685f757-c66d-4095-873c-3dab2e20b02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:01.296372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 680/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:12:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:05.319268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8283-523e-4534-8734-614f762f8b54 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:05.320331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8283-523e-4534-8734-614f762f8b54 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:05.320852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f8283-523e-4534-8734-614f762f8b54 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:05.322371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 672/1352] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:05.327972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8d575-90a5-4532-a68f-00694d0c0089 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:05.328447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8d575-90a5-4532-a68f-00694d0c0089 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:05.328677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a8d575-90a5-4532-a68f-00694d0c0089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:05.347894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a3a8d575-90a5-4532-a68f-00694d0c0089 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:05.349440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a3a8d575-90a5-4532-a68f-00694d0c0089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:05.359251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 681/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:12:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:12:05.580408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=13 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:06.916217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bbd-22f3-4782-9110-8fc9bdab62c0 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:06.916880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bbd-22f3-4782-9110-8fc9bdab62c0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:06.916880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869bbd-22f3-4782-9110-8fc9bdab62c0 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:06.918138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 673/1354] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:06.923100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e58b1-6cba-43c0-ac38-a44780190671 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:06.923513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e58b1-6cba-43c0-ac38-a44780190671 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:06.923694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e58b1-6cba-43c0-ac38-a44780190671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:06.942031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6b4e58b1-6cba-43c0-ac38-a44780190671 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:06.943224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6b4e58b1-6cba-43c0-ac38-a44780190671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:06.951839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 682/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:12:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:12:10.850822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331be20-ab96-44f1-b95f-9188b7273355 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:10.852341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331be20-ab96-44f1-b95f-9188b7273355 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:10.852860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331be20-ab96-44f1-b95f-9188b7273355 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:10.857469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 674/1356] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:10.859846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e52e-302b-41ac-bb07-fc13b155c511 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:10.860631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e52e-302b-41ac-bb07-fc13b155c511 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:10.860919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d494e52e-302b-41ac-bb07-fc13b155c511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:10.881775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d494e52e-302b-41ac-bb07-fc13b155c511 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:10.882975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d494e52e-302b-41ac-bb07-fc13b155c511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:10.890841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 683/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:10.946239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121bcc-ca18-4cca-8697-bac9cd69dadb None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:10.946916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121bcc-ca18-4cca-8697-bac9cd69dadb None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:10.947687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8121bcc-ca18-4cca-8697-bac9cd69dadb None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:10.949418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 675/1358] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 15:12:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:10.959144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf41c7-c593-44a1-b6c6-366c2e3705ff None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:10.960035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf41c7-c593-44a1-b6c6-366c2e3705ff None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:10.961173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcf41c7-c593-44a1-b6c6-366c2e3705ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:11.016623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:11.026024 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2bcf41c7-c593-44a1-b6c6-366c2e3705ff None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:11.027984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2bcf41c7-c593-44a1-b6c6-366c2e3705ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:11.122205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 684/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:11.130248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8121bcc-ca18-4cca-8697-bac9cd69dadb None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:11.145574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8121bcc-ca18-4cca-8697-bac9cd69dadb None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:11.151367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:11.152111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:11.152489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:11.152709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:11.162595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:11.164220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273f6f3-4c48-4709-a963-673439dbf50c service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:11.175977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 676/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:12:13.606216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dca1a-7136-42bc-8810-521f68823c7e None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:13.606741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dca1a-7136-42bc-8810-521f68823c7e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:13.606741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dca1a-7136-42bc-8810-521f68823c7e None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:13.608235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 685/1361] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:13.613824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a205e1-7e12-4d74-83e5-8a86e6927127 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:13.614242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a205e1-7e12-4d74-83e5-8a86e6927127 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:13.614384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a205e1-7e12-4d74-83e5-8a86e6927127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:13.631956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-94a205e1-7e12-4d74-83e5-8a86e6927127 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:13.633152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94a205e1-7e12-4d74-83e5-8a86e6927127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:13.643704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 677/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:13.687478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dca1a-7136-42bc-8810-521f68823c7e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:13.703250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dca1a-7136-42bc-8810-521f68823c7e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:13.709439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:13.710305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:13.710667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:13.710953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:13.719811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:13.722087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1f625-b6e8-4fc1-b3c8-961a706e8a5f service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:13.733858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 686/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:12:16.286399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802dc05-1373-4fe7-9da8-23ad5abcccd9 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:16.286874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802dc05-1373-4fe7-9da8-23ad5abcccd9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:16.286969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802dc05-1373-4fe7-9da8-23ad5abcccd9 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:16.288348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 678/1364] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:16.293680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e65f0-df68-4479-a81d-cb1b9d5120bd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:16.294110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e65f0-df68-4479-a81d-cb1b9d5120bd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:16.294250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e65f0-df68-4479-a81d-cb1b9d5120bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:16.317354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-220e65f0-df68-4479-a81d-cb1b9d5120bd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:16.318539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-220e65f0-df68-4479-a81d-cb1b9d5120bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:16.327453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 687/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:16.379340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f47bf-ab34-49c3-971c-1049e4ccf562 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:16.379622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f47bf-ab34-49c3-971c-1049e4ccf562 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:16.380240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f47bf-ab34-49c3-971c-1049e4ccf562 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:16.381661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 679/1366] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 15:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:16.387428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fb2fc-9573-4e03-88cf-d717fc372222 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:16.387714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fb2fc-9573-4e03-88cf-d717fc372222 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:16.387990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fb2fc-9573-4e03-88cf-d717fc372222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:16.407168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6b1fb2fc-9573-4e03-88cf-d717fc372222 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:16.408342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6b1fb2fc-9573-4e03-88cf-d717fc372222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:16.417496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 688/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:16.425224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f47bf-ab34-49c3-971c-1049e4ccf562 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:16.439792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f47bf-ab34-49c3-971c-1049e4ccf562 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:16.445144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:16.446509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:16.446797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:16.447074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:16.455726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:16.457361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bf22a-ae35-4936-be24-8f85223819b2 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:16.466595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 680/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:18.745725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c09-2f2f-46e2-a38f-9c9d43e3e996 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:18.747056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c09-2f2f-46e2-a38f-9c9d43e3e996 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:18.747694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c09-2f2f-46e2-a38f-9c9d43e3e996 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:18.749968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 689/1369] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:18.755698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb7a6-de9f-49d1-8f14-7928997cfbfe None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:18.756158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb7a6-de9f-49d1-8f14-7928997cfbfe None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:18.756257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58feb7a6-de9f-49d1-8f14-7928997cfbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:18.774612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-58feb7a6-de9f-49d1-8f14-7928997cfbfe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:18.775800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-58feb7a6-de9f-49d1-8f14-7928997cfbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:18.783909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 681/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:18.829320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c09-2f2f-46e2-a38f-9c9d43e3e996 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:18.844013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c09-2f2f-46e2-a38f-9c9d43e3e996 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:18.849578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:18.850220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:18.850508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:18.850784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:18.860269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:18.862067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9647e6a-aaa7-4d35-a37b-14e645a626d7 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:18.872083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 690/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:19.087709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37db9ba-0dd0-4787-9642-d42381c00512 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:19.088002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37db9ba-0dd0-4787-9642-d42381c00512 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:19.088284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37db9ba-0dd0-4787-9642-d42381c00512 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:19.089764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 682/1372] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:19.096651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fc485-4533-4bee-84d2-16c4806b7585 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:19.096950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fc485-4533-4bee-84d2-16c4806b7585 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:19.097267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fc485-4533-4bee-84d2-16c4806b7585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:19.117362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ee1fc485-4533-4bee-84d2-16c4806b7585 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:19.118610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ee1fc485-4533-4bee-84d2-16c4806b7585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:19.132382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 691/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:19.446845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9526f-f22c-4c57-a77d-539b6141f59f None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:19.446845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9526f-f22c-4c57-a77d-539b6141f59f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:19.446845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9526f-f22c-4c57-a77d-539b6141f59f None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:19.447933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 683/1374] 10.222.0.45 () {54 vars in 963 bytes} [Tue May 24 15:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:12:19.455952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4389b4-8d3c-4a0c-889e-529c55b9f504 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:19.456403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4389b4-8d3c-4a0c-889e-529c55b9f504 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:19.456782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4389b4-8d3c-4a0c-889e-529c55b9f504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:19.507335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1f4389b4-8d3c-4a0c-889e-529c55b9f504 None None] MFA Rules not processed for user `94ce1868b4ac4d2a8273867aab1bfe80`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:19.508716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1f4389b4-8d3c-4a0c-889e-529c55b9f504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:19.575622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 692/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Tue May 24 15:12:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:19.583175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9526f-f22c-4c57-a77d-539b6141f59f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:19.593914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9526f-f22c-4c57-a77d-539b6141f59f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:19.598147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:19.598910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:19.599297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:19.599674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:19.609378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:19.610937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3470801-535d-4e09-a320-a99b30bd1877 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:19.621373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 684/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:21.168156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604f20-c4b2-45ec-9b55-8f56e81b7c82 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:21.168712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604f20-c4b2-45ec-9b55-8f56e81b7c82 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:21.168712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a604f20-c4b2-45ec-9b55-8f56e81b7c82 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:21.170259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 693/1377] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:21.176231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb66d-b4a1-475b-ab65-e3958c212a1f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:21.176746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb66d-b4a1-475b-ab65-e3958c212a1f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:21.176836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23bb66d-b4a1-475b-ab65-e3958c212a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:21.195901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f23bb66d-b4a1-475b-ab65-e3958c212a1f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:21.197160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f23bb66d-b4a1-475b-ab65-e3958c212a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:21.205341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 685/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:21.251124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a604f20-c4b2-45ec-9b55-8f56e81b7c82 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:21.266840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a604f20-c4b2-45ec-9b55-8f56e81b7c82 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:21.272413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:21.273081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:21.273368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:21.273643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:21.283049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:21.284767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e445cecb-accc-4b3e-85d7-a348060f9a35 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:21.289344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:21.297157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 694/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:23.534918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec21e2-364b-4a71-984a-e80b3d670279 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:23.535484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec21e2-364b-4a71-984a-e80b3d670279 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:23.535484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec21e2-364b-4a71-984a-e80b3d670279 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:23.536872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 686/1380] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:23.542159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381787f1-2a91-43ae-87d3-b2733dcae01c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:23.542558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381787f1-2a91-43ae-87d3-b2733dcae01c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:23.542714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381787f1-2a91-43ae-87d3-b2733dcae01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:23.560763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-381787f1-2a91-43ae-87d3-b2733dcae01c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:23.561938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-381787f1-2a91-43ae-87d3-b2733dcae01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:23.570300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 695/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:23.614108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec21e2-364b-4a71-984a-e80b3d670279 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:23.629239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec21e2-364b-4a71-984a-e80b3d670279 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:23.634564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:23.635211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:23.635496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:23.635769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:23.644150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:23.645787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78c0999-e852-4d1d-9f54-f6442f21db29 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:23.655497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 687/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:25.896521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f87b4-7b56-42b9-a05d-225207935a78 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:25.897039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f87b4-7b56-42b9-a05d-225207935a78 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:25.897039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f87b4-7b56-42b9-a05d-225207935a78 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:25.898553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 696/1383] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:25.904160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d5b4-cdc5-480c-91ad-2caced00ebd6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:25.904628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d5b4-cdc5-480c-91ad-2caced00ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:25.904775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46d5b4-cdc5-480c-91ad-2caced00ebd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:25.923075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ec46d5b4-cdc5-480c-91ad-2caced00ebd6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:25.925033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ec46d5b4-cdc5-480c-91ad-2caced00ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:25.937306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 688/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:25.975752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f87b4-7b56-42b9-a05d-225207935a78 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:25.992427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281f87b4-7b56-42b9-a05d-225207935a78 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:25.998618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:25.999313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:25.999598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:25.999877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:26.010798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:26.012522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f0f0b2-9b84-4bfc-a001-01e776069894 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:26.023948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 697/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:28.239484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779874-c1bc-46a5-be23-04dacc0bdaba None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:28.240092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779874-c1bc-46a5-be23-04dacc0bdaba None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:28.240092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779874-c1bc-46a5-be23-04dacc0bdaba None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:28.241478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 689/1386] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:28.246872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7482-63b3-450c-8d09-75d49da62f94 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:28.247310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7482-63b3-450c-8d09-75d49da62f94 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:28.247460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630e7482-63b3-450c-8d09-75d49da62f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:28.265629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-630e7482-63b3-450c-8d09-75d49da62f94 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:28.266804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-630e7482-63b3-450c-8d09-75d49da62f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:28.274493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 698/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:28.320030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779874-c1bc-46a5-be23-04dacc0bdaba None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:28.334324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779874-c1bc-46a5-be23-04dacc0bdaba None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:28.339697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:28.340356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:28.340644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:28.340921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:28.349809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:28.351405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd93e01-cae7-4e67-a38c-64bd3b8d908e service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:28.361988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 690/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:30.628868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aecbf-707d-41b1-ac32-c4cb15344667 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:30.629558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aecbf-707d-41b1-ac32-c4cb15344667 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:30.629558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19aecbf-707d-41b1-ac32-c4cb15344667 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:30.630926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 699/1389] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:12:30.636431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646977-a54b-4ee8-ba2e-74ca6ff50cb5 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:30.636841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646977-a54b-4ee8-ba2e-74ca6ff50cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:30.636986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc646977-a54b-4ee8-ba2e-74ca6ff50cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:30.654724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-dc646977-a54b-4ee8-ba2e-74ca6ff50cb5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:30.655898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dc646977-a54b-4ee8-ba2e-74ca6ff50cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:30.664252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 691/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:30.710547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19aecbf-707d-41b1-ac32-c4cb15344667 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:30.725870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19aecbf-707d-41b1-ac32-c4cb15344667 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:30.731574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:30.732215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:30.732530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:30.732812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:30.741631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:30.743232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa05c-e1da-4f6c-b307-cd118f5a74dc service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:30.752741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 700/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:33.055005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16236-ef80-47d8-b805-7e363f709a05 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:33.055642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16236-ef80-47d8-b805-7e363f709a05 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:33.055642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16236-ef80-47d8-b805-7e363f709a05 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:33.056940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 692/1392] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:33.062524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813560a2-99c5-4d53-990f-f8d628ff204d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:33.062955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813560a2-99c5-4d53-990f-f8d628ff204d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:33.063111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813560a2-99c5-4d53-990f-f8d628ff204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:33.081112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-813560a2-99c5-4d53-990f-f8d628ff204d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:33.082305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-813560a2-99c5-4d53-990f-f8d628ff204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:33.090243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 701/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:33.144455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e16236-ef80-47d8-b805-7e363f709a05 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:33.159482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e16236-ef80-47d8-b805-7e363f709a05 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:33.165512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:33.166191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:33.166473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:33.166748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:33.182361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:33.184068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbf498-b3ca-42a0-95a3-e8f3c46326ea service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:33.193803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 693/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:35.176337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3c78-38a1-4413-9449-1ff16c3150bf None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.176860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3c78-38a1-4413-9449-1ff16c3150bf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.176860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb3c78-38a1-4413-9449-1ff16c3150bf None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.178167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 702/1395] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:35.183299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712a7c-533e-4e85-aec7-d1cf349ba601 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.183713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712a7c-533e-4e85-aec7-d1cf349ba601 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.183852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb712a7c-533e-4e85-aec7-d1cf349ba601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.201266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bb712a7c-533e-4e85-aec7-d1cf349ba601 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:35.202426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bb712a7c-533e-4e85-aec7-d1cf349ba601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:35.210106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 694/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:12:35.266938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74108ed-c0c8-4d6b-b00c-16b0e93bc3c9 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.267284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74108ed-c0c8-4d6b-b00c-16b0e93bc3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.267560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74108ed-c0c8-4d6b-b00c-16b0e93bc3c9 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.269046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 703/1397] 10.222.0.45 () {54 vars in 962 bytes} [Tue May 24 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:35.274131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a1bd5-d4f1-4dc6-9192-252a938df091 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.274484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a1bd5-d4f1-4dc6-9192-252a938df091 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.274825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a1bd5-d4f1-4dc6-9192-252a938df091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.323408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1d1a1bd5-d4f1-4dc6-9192-252a938df091 None None] MFA Rules not processed for user `847292d322f6483ba5ff02294ad409dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:35.324611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1d1a1bd5-d4f1-4dc6-9192-252a938df091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:35.384664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 695/1398] 10.222.0.45 () {58 vars in 1059 bytes} [Tue May 24 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:35.392867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74108ed-c0c8-4d6b-b00c-16b0e93bc3c9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:35.403329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74108ed-c0c8-4d6b-b00c-16b0e93bc3c9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:35.406809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:35.407481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.408395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.408691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.413041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734cc36-addc-4769-8bce-397a0aaf45b1 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.413331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734cc36-addc-4769-8bce-397a0aaf45b1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.413606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7734cc36-addc-4769-8bce-397a0aaf45b1 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.414967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 696/1399] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:12:35.418211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:35.419634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:35.420880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5e9a-6c3e-43cd-bd06-2db638c6f998 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.421171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5e9a-6c3e-43cd-bd06-2db638c6f998 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.421448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b5e9a-6c3e-43cd-bd06-2db638c6f998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.429300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 704/1400] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:35.439152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0b0b5e9a-6c3e-43cd-bd06-2db638c6f998 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:35.440350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0b0b5e9a-6c3e-43cd-bd06-2db638c6f998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:35.448099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 697/1401] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:12:35.492284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:35.504462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932a5cc5-a795-447d-9ae7-e66b771624b5 service glance] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:35.509936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:35.510683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.510977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.511256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.519804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:35.521438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e095dce-e67f-44c4-9cbd-f01e99a5839f service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:35.530769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 705/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:35.969763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9822f-fdc7-42f6-ac06-168cccee3ac9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:35.970078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9822f-fdc7-42f6-ac06-168cccee3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:35.970396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e9822f-fdc7-42f6-ac06-168cccee3ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:35.975541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:36.019811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-19e9822f-fdc7-42f6-ac06-168cccee3ac9 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:36.021115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-19e9822f-fdc7-42f6-ac06-168cccee3ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:36.089682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 698/1403] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:36.132576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f215-f6d6-4452-a669-fd69364e6ef4 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:36.132954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f215-f6d6-4452-a669-fd69364e6ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:36.133246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28f215-f6d6-4452-a669-fd69364e6ef4 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:36.134619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 706/1404] 10.222.0.45 () {54 vars in 942 bytes} [Tue May 24 15:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:36.140904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168387-dab0-4dc8-8980-328c9026bc6a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:36.141294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168387-dab0-4dc8-8980-328c9026bc6a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:36.141654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5168387-dab0-4dc8-8980-328c9026bc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:36.190353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f5168387-dab0-4dc8-8980-328c9026bc6a None None] MFA Rules not processed for user `ec47b745ff5a4ced9b47ed5ecd2b28cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:36.191538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f5168387-dab0-4dc8-8980-328c9026bc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:36.261474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 699/1405] 10.222.0.45 () {58 vars in 1039 bytes} [Tue May 24 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:36.271469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28f215-f6d6-4452-a669-fd69364e6ef4 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:36.284846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28f215-f6d6-4452-a669-fd69364e6ef4 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:36.289688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:36.290332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:36.290613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:36.290892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:36.299834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:36.301441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4787b90-5a70-4d77-8261-c4f1ee638215 service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:36.309491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 707/1406] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:12:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:12:37.766650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146c28-9aa9-4575-a385-dbf7ff173d04 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:37.767748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146c28-9aa9-4575-a385-dbf7ff173d04 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:37.769190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9146c28-9aa9-4575-a385-dbf7ff173d04 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:37.772339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 700/1407] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:37.775007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0b15-b102-4559-bdb5-c9f5761a1645 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:37.775422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0b15-b102-4559-bdb5-c9f5761a1645 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:37.775564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0b15-b102-4559-bdb5-c9f5761a1645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:37.799215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-199b0b15-b102-4559-bdb5-c9f5761a1645 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:37.800608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-199b0b15-b102-4559-bdb5-c9f5761a1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:37.809599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 708/1408] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:37.854659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146c28-9aa9-4575-a385-dbf7ff173d04 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:12:37.869853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9146c28-9aa9-4575-a385-dbf7ff173d04 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:12:37.876649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:12:37.877556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:37.877842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:37.878117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:37.887598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:12:37.889335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d10cb8-331f-4915-9a5e-97b4faad44b1 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:12:37.903814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 701/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:12:47.789692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=43 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:47.895512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:12:59.162125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574f6f-90ea-491c-8d1c-a813d81c10f1 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:59.162735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574f6f-90ea-491c-8d1c-a813d81c10f1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:59.162735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79574f6f-90ea-491c-8d1c-a813d81c10f1 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:59.163975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 709/1410] 10.222.0.45 () {54 vars in 918 bytes} [Tue May 24 15:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:59.169115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d370244-97ec-4a6d-b0a0-81e0aa60dda0 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:59.169607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d370244-97ec-4a6d-b0a0-81e0aa60dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:59.169714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d370244-97ec-4a6d-b0a0-81e0aa60dda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:59.187110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5d370244-97ec-4a6d-b0a0-81e0aa60dda0 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:59.188284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5d370244-97ec-4a6d-b0a0-81e0aa60dda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:59.197213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 702/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Tue May 24 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:12:59.211922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a4f-eda9-4887-9cb2-b3ebb68f1b0b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:59.212242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a4f-eda9-4887-9cb2-b3ebb68f1b0b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:59.212552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a4f-eda9-4887-9cb2-b3ebb68f1b0b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:59.213945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 710/1412] 10.222.0.45 () {54 vars in 918 bytes} [Tue May 24 15:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:12:59.218264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012bfa-f94b-4ef9-864d-245461943255 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:12:59.218546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012bfa-f94b-4ef9-864d-245461943255 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:12:59.218818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67012bfa-f94b-4ef9-864d-245461943255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:12:59.236432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-67012bfa-f94b-4ef9-864d-245461943255 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:12:59.237592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-67012bfa-f94b-4ef9-864d-245461943255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:12:59.246430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 703/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Tue May 24 15:12:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:13:01.935862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46700f-49d4-4719-8690-f2a890da086f None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:01.936461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46700f-49d4-4719-8690-f2a890da086f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:01.936461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46700f-49d4-4719-8690-f2a890da086f None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:01.937782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:01.943626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499576b-fa89-40a6-9373-30fc59e127d3 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:01.944092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499576b-fa89-40a6-9373-30fc59e127d3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:01.944187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499576b-fa89-40a6-9373-30fc59e127d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:01.961397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c499576b-fa89-40a6-9373-30fc59e127d3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:01.962596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c499576b-fa89-40a6-9373-30fc59e127d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:01.970490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:02.013200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46700f-49d4-4719-8690-f2a890da086f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:02.027427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46700f-49d4-4719-8690-f2a890da086f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:02.033151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:02.033777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:02.034051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:02.034321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:02.042260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:02.043829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99c194-2439-4d04-bd4a-1b7623724a0c service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:02.053571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 712/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:03.012672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bc8e-79be-4121-97df-e185ee7560af None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:03.013215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bc8e-79be-4121-97df-e185ee7560af None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:03.013215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192bc8e-79be-4121-97df-e185ee7560af None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:03.014579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:03.019440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13735-5f03-4d2b-8479-ae9c51c825cb None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:03.019845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13735-5f03-4d2b-8479-ae9c51c825cb None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:03.019962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d13735-5f03-4d2b-8479-ae9c51c825cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:03.037851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-88d13735-5f03-4d2b-8479-ae9c51c825cb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:03.039034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-88d13735-5f03-4d2b-8479-ae9c51c825cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:03.046684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:03.208009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192bc8e-79be-4121-97df-e185ee7560af None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:03.222593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192bc8e-79be-4121-97df-e185ee7560af None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:03.227890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:03.228558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:03.228854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:03.229115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:03.237304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:03.238893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0919a-cc0f-4d2f-be27-779d1abe457f service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:03.248223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 706/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:04.238808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e285-6c30-4824-8652-5f6623c2eae7 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:04.239359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e285-6c30-4824-8652-5f6623c2eae7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:04.239359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e285-6c30-4824-8652-5f6623c2eae7 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:04.240793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:04.245606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4787c0-85c4-4782-b51e-4ecb0a7b1da6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:04.246010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4787c0-85c4-4782-b51e-4ecb0a7b1da6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:04.246171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4787c0-85c4-4782-b51e-4ecb0a7b1da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:04.264283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-da4787c0-85c4-4782-b51e-4ecb0a7b1da6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:04.265498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-da4787c0-85c4-4782-b51e-4ecb0a7b1da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:04.273279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:04.443953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3e285-6c30-4824-8652-5f6623c2eae7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:04.457957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3e285-6c30-4824-8652-5f6623c2eae7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:04.463361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:04.463997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:04.464283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:04.464589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:04.473116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:04.474707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18685aad-133a-41ba-bfb5-0c0595f14599 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:04.484349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 715/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:13:05.455475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6d502-fda7-4be3-94db-65c7e1118431 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:05.456049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6d502-fda7-4be3-94db-65c7e1118431 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:05.456049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6d502-fda7-4be3-94db-65c7e1118431 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:05.457429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:05.462273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ccb3-0f49-4b90-9d6f-dfd6d851ce99 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:05.462674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ccb3-0f49-4b90-9d6f-dfd6d851ce99 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:05.462832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96ccb3-0f49-4b90-9d6f-dfd6d851ce99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:05.481494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6e96ccb3-0f49-4b90-9d6f-dfd6d851ce99 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:05.482692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6e96ccb3-0f49-4b90-9d6f-dfd6d851ce99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:05.490457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:05.636763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6d502-fda7-4be3-94db-65c7e1118431 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:05.651428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6d502-fda7-4be3-94db-65c7e1118431 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:05.656695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:05.657927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:05.658208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:05.658479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:05.666374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:05.667959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1816f8-c03a-48f0-8604-2d91ed6df908 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:05.677592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 709/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:06.577091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7887-863f-4d8d-8225-c81d41d48cd7 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:06.577673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7887-863f-4d8d-8225-c81d41d48cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:06.577673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a7887-863f-4d8d-8225-c81d41d48cd7 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:06.579014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:06.583852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4927559-a5eb-42f5-bf93-fa4dd3d66090 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:06.584252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4927559-a5eb-42f5-bf93-fa4dd3d66090 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:06.584437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4927559-a5eb-42f5-bf93-fa4dd3d66090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:06.602392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a4927559-a5eb-42f5-bf93-fa4dd3d66090 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:06.603574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a4927559-a5eb-42f5-bf93-fa4dd3d66090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:06.612111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:06.764823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a7887-863f-4d8d-8225-c81d41d48cd7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:06.779371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a7887-863f-4d8d-8225-c81d41d48cd7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:06.785430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:06.786078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:06.786358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:06.786632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:06.794946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:06.796556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc2501-04d9-44f0-8da3-14b18bf166f0 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:06.806627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 718/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:07.650868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad2d0-ab6f-4adc-aa88-14742dd6d088 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:07.651449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad2d0-ab6f-4adc-aa88-14742dd6d088 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:07.651562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad2d0-ab6f-4adc-aa88-14742dd6d088 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:07.652954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:07.658045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06cbda-7f16-4bb8-a751-3d2cbbcfab82 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:07.658458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06cbda-7f16-4bb8-a751-3d2cbbcfab82 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:07.658607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f06cbda-7f16-4bb8-a751-3d2cbbcfab82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:07.676737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2f06cbda-7f16-4bb8-a751-3d2cbbcfab82 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:07.677932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2f06cbda-7f16-4bb8-a751-3d2cbbcfab82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:07.685674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:07.835038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad2d0-ab6f-4adc-aa88-14742dd6d088 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:07.849031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad2d0-ab6f-4adc-aa88-14742dd6d088 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:07.854302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:07.854924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:07.855202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:07.855474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:07.864532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:07.866122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060155d3-3f5b-4ed3-a940-0cb83f75f5b7 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:07.875418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 712/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:08.724416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d39-e5dd-4d23-afff-53b9306904b2 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:08.724983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d39-e5dd-4d23-afff-53b9306904b2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:08.724983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b2d39-e5dd-4d23-afff-53b9306904b2 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:08.726354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:08.731432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d944d-5d24-4b14-9735-372f6b185052 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:08.731842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d944d-5d24-4b14-9735-372f6b185052 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:08.732012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d944d-5d24-4b14-9735-372f6b185052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:08.749512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-069d944d-5d24-4b14-9735-372f6b185052 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:08.750684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-069d944d-5d24-4b14-9735-372f6b185052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:08.758826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:08.904898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b2d39-e5dd-4d23-afff-53b9306904b2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:08.919379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b2d39-e5dd-4d23-afff-53b9306904b2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:08.924753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:08.925412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:08.925695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:08.925977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:08.934103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:08.935688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8053ae6a-b853-4927-b3aa-fdecbf411430 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:08.945586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 721/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:09.779318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0182-4a88-4d88-bcfc-55257cf47669 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:09.779875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0182-4a88-4d88-bcfc-55257cf47669 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:09.779875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d0182-4a88-4d88-bcfc-55257cf47669 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:09.781299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:09.786475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa063b-e479-432f-a366-a92db6e15cfd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:09.786875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa063b-e479-432f-a366-a92db6e15cfd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:09.787032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa063b-e479-432f-a366-a92db6e15cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:09.804737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9cfa063b-e479-432f-a366-a92db6e15cfd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:09.805929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9cfa063b-e479-432f-a366-a92db6e15cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:09.814158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:09.964827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d0182-4a88-4d88-bcfc-55257cf47669 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:09.978827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d0182-4a88-4d88-bcfc-55257cf47669 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:09.984477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:09.985110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:09.985884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:09.986158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:09.994105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:09.995696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1043f2-b893-44ad-9d1a-667098e9ca0d service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:10.005519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 715/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:10.877190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a142c9-bd07-4d03-a141-5e857cd66ddc None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:10.877675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a142c9-bd07-4d03-a141-5e857cd66ddc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:10.877799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a142c9-bd07-4d03-a141-5e857cd66ddc None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:10.879113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:10.884243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cab83-e560-4493-b674-a16369ae2d79 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:10.884626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cab83-e560-4493-b674-a16369ae2d79 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:10.884705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cab83-e560-4493-b674-a16369ae2d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:10.902359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1c4cab83-e560-4493-b674-a16369ae2d79 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:10.903537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1c4cab83-e560-4493-b674-a16369ae2d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:10.911192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:11.059185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a142c9-bd07-4d03-a141-5e857cd66ddc None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:11.073602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a142c9-bd07-4d03-a141-5e857cd66ddc None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:11.079247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:11.079892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:11.080170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:11.080473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:11.088670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:11.090242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15daf1-6c2f-41e2-aba5-8dc9097c7c67 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:11.099878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 724/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:13.158372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:13.158871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:13.158871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:13.160219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:13.165447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049bfc-9f31-4523-b77d-e92c1432ef59 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:13.165877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049bfc-9f31-4523-b77d-e92c1432ef59 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:13.166034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7049bfc-9f31-4523-b77d-e92c1432ef59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:13.183323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c7049bfc-9f31-4523-b77d-e92c1432ef59 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:13.184650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c7049bfc-9f31-4523-b77d-e92c1432ef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:13.192430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:13.230939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:13.232942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:13.245699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f662a54-04d9-4f05-993f-b58077effe10 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:13.249205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:13.249832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:13.250140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:13.250415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:13.259831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:13.261308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79876ee8-93ae-4bf0-9bfa-0b9399376dad service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:13.271087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 718/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:15.585658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4e2-7aac-4320-a5ff-46798f76d31d None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:15.586202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4e2-7aac-4320-a5ff-46798f76d31d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:15.586202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857e4e2-7aac-4320-a5ff-46798f76d31d None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:15.587584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:15.593091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562540-0fa8-4c3a-8744-75b99bc45d08 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:15.593542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562540-0fa8-4c3a-8744-75b99bc45d08 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:15.593647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562540-0fa8-4c3a-8744-75b99bc45d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:15.613262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9b562540-0fa8-4c3a-8744-75b99bc45d08 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:15.614432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b562540-0fa8-4c3a-8744-75b99bc45d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:15.622116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:15.660735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857e4e2-7aac-4320-a5ff-46798f76d31d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:15.671505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857e4e2-7aac-4320-a5ff-46798f76d31d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:15.674999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:15.675628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:15.675907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:15.676179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:15.685016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:15.686376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e2fb2-63b7-4968-bce9-d87c90b11e27 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:15.696039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 727/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:18.268760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71820e0-a7d0-402f-9c03-728cfcf79ef6 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:18.269645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71820e0-a7d0-402f-9c03-728cfcf79ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:18.270110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71820e0-a7d0-402f-9c03-728cfcf79ef6 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:18.271493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:18.277008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39647-f856-44f7-b4de-0a6c1e1b128f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:18.277455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39647-f856-44f7-b4de-0a6c1e1b128f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:18.277598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d39647-f856-44f7-b4de-0a6c1e1b128f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:18.298813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b8d39647-f856-44f7-b4de-0a6c1e1b128f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:18.300010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b8d39647-f856-44f7-b4de-0a6c1e1b128f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:18.307990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:18.346158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71820e0-a7d0-402f-9c03-728cfcf79ef6 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:18.356838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71820e0-a7d0-402f-9c03-728cfcf79ef6 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:18.360777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:18.361404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:18.361686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:18.361958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:18.370891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:18.372371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25851ed-baca-45b6-b7e7-dfffa0beb33c service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:18.382337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 721/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:20.696012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e7f3-3c83-4cc7-8804-0cf681fa167d None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:20.696660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e7f3-3c83-4cc7-8804-0cf681fa167d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:20.696660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7e7f3-3c83-4cc7-8804-0cf681fa167d None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:20.697941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:20.703128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124bf08-b18f-4b38-b8ca-a38897fe031a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:20.703539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124bf08-b18f-4b38-b8ca-a38897fe031a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:20.703677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124bf08-b18f-4b38-b8ca-a38897fe031a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:20.722696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e124bf08-b18f-4b38-b8ca-a38897fe031a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:20.723894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e124bf08-b18f-4b38-b8ca-a38897fe031a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:20.731929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:20.770596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7e7f3-3c83-4cc7-8804-0cf681fa167d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:20.781162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7e7f3-3c83-4cc7-8804-0cf681fa167d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:20.784635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:20.785263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:20.785542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:20.785809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:20.794357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:20.795778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428cd459-2f5f-4c89-afe7-ebb39dddb6f5 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:20.804872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 730/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:23.060321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115933d-1802-4f86-867b-88d28a58bfd3 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:23.060869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115933d-1802-4f86-867b-88d28a58bfd3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:23.060869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115933d-1802-4f86-867b-88d28a58bfd3 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:23.062245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:23.068030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419e80-2c45-4cf1-820d-a85287090945 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:23.068497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419e80-2c45-4cf1-820d-a85287090945 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:23.068607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5419e80-2c45-4cf1-820d-a85287090945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:23.086936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c5419e80-2c45-4cf1-820d-a85287090945 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:23.088122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c5419e80-2c45-4cf1-820d-a85287090945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:23.096100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:13:23.134462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115933d-1802-4f86-867b-88d28a58bfd3 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:23.145610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115933d-1802-4f86-867b-88d28a58bfd3 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:23.149049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:23.149679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:23.149961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:23.150506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:23.159289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:23.160777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f79456-a782-4579-9b21-ff842e2a5b32 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:23.169948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 724/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:25.397547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25332176-e085-4f74-a3ef-2dac201cbac3 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:25.398086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25332176-e085-4f74-a3ef-2dac201cbac3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:25.398086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25332176-e085-4f74-a3ef-2dac201cbac3 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:25.399508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:25.404786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e10-b605-4b22-8e59-6fef64b43350 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:25.405206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e10-b605-4b22-8e59-6fef64b43350 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:25.405348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7e10-b605-4b22-8e59-6fef64b43350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:25.422699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9cfd7e10-b605-4b22-8e59-6fef64b43350 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:25.423886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9cfd7e10-b605-4b22-8e59-6fef64b43350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:25.431622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:25.469643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25332176-e085-4f74-a3ef-2dac201cbac3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:25.480315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25332176-e085-4f74-a3ef-2dac201cbac3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:25.484064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:25.484723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:25.485036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:25.485313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:25.494156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:25.495584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9ab38-ad05-455a-b7c0-b1f95d52d01c service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:25.505267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 733/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:27.743753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79d77-2010-49ba-bac9-5c547aca2aad None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:27.744346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79d77-2010-49ba-bac9-5c547aca2aad None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:27.744346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c79d77-2010-49ba-bac9-5c547aca2aad None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:27.745702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:27.751497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf0443-bcd3-444b-a9e8-3d9998714d89 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:27.751896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf0443-bcd3-444b-a9e8-3d9998714d89 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:27.752086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf0443-bcd3-444b-a9e8-3d9998714d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:27.770001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c3bf0443-bcd3-444b-a9e8-3d9998714d89 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:27.771182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c3bf0443-bcd3-444b-a9e8-3d9998714d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:27.779018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:13:27.816646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c79d77-2010-49ba-bac9-5c547aca2aad None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:27.827733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c79d77-2010-49ba-bac9-5c547aca2aad None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:27.831168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:27.831794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:27.832069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:27.832368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:27.841051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:27.842482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd529c7c-380d-4065-a952-842159bfc736 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:27.851995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 727/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:30.064646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3c5f-887a-495b-abd5-8a5aa195fee4 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:30.065179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3c5f-887a-495b-abd5-8a5aa195fee4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:30.065179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f3c5f-887a-495b-abd5-8a5aa195fee4 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:30.066570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:30.071909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8802ad-31c9-468b-8a24-682b63c07db2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:30.072321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8802ad-31c9-468b-8a24-682b63c07db2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:30.072497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8802ad-31c9-468b-8a24-682b63c07db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:30.078514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=79 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:13:30.091434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ce8802ad-31c9-468b-8a24-682b63c07db2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:30.092663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ce8802ad-31c9-468b-8a24-682b63c07db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:30.104585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:30.141897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f3c5f-887a-495b-abd5-8a5aa195fee4 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:30.147161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=73 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:13:30.153560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f3c5f-887a-495b-abd5-8a5aa195fee4 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:30.157640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:30.158267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:30.158545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:30.158815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:30.167503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:30.168959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bbde9e-dae6-4b54-8c5b-0be6a125babd service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:30.178134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 736/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:32.384609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1197bf-04cf-49ef-b3c2-a7d12e5939d2 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:32.385138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1197bf-04cf-49ef-b3c2-a7d12e5939d2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:32.385138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1197bf-04cf-49ef-b3c2-a7d12e5939d2 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:32.386459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:32.392195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f3ad8-9011-4b0c-afe5-e147315c3087 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:32.392665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f3ad8-9011-4b0c-afe5-e147315c3087 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:32.392761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f3ad8-9011-4b0c-afe5-e147315c3087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:32.411719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7d4f3ad8-9011-4b0c-afe5-e147315c3087 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:32.413264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7d4f3ad8-9011-4b0c-afe5-e147315c3087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:32.421817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:32.458037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1197bf-04cf-49ef-b3c2-a7d12e5939d2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:32.468967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1197bf-04cf-49ef-b3c2-a7d12e5939d2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:32.473023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:32.473651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:32.473937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:32.474208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:32.483179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:32.484668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace88ec0-997f-4642-9495-5bccdd44efac service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:32.493860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 730/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:34.689481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182fac4-9cd2-42f3-92ad-184457d25e0d None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:34.690019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182fac4-9cd2-42f3-92ad-184457d25e0d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:34.690019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182fac4-9cd2-42f3-92ad-184457d25e0d None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:34.691438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:13:34.697024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bccd5d-c962-4ab7-ba5d-92870a91e31f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:34.697459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bccd5d-c962-4ab7-ba5d-92870a91e31f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:34.697584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bccd5d-c962-4ab7-ba5d-92870a91e31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:34.714778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-14bccd5d-c962-4ab7-ba5d-92870a91e31f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:13:34.715957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-14bccd5d-c962-4ab7-ba5d-92870a91e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:13:34.723685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:13:34.762332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182fac4-9cd2-42f3-92ad-184457d25e0d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:13:34.776983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182fac4-9cd2-42f3-92ad-184457d25e0d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:13:34.783851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:13:34.784505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:13:34.784784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:13:34.785055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:13:34.795407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:13:34.796872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adf6f9-a79e-466f-898e-4f188e57d20e service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:13:34.806552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 739/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:13:44.709720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=11 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:13:44.802445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=14 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:14:07.694614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e492-a6d8-45df-84c9-b5ee9c7d6423 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:07.695250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e492-a6d8-45df-84c9-b5ee9c7d6423 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:07.695250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82e492-a6d8-45df-84c9-b5ee9c7d6423 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:07.696630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 732/1471] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:07.701873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b8d7-e241-4279-94d1-7bc7b690c13a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:07.702292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b8d7-e241-4279-94d1-7bc7b690c13a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:07.702501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534b8d7-e241-4279-94d1-7bc7b690c13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:07.721026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2534b8d7-e241-4279-94d1-7bc7b690c13a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:07.722212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2534b8d7-e241-4279-94d1-7bc7b690c13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:07.729897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 740/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:07.776124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3356f-bd3b-4362-9ada-2b0457f9f6d8 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:07.776450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3356f-bd3b-4362-9ada-2b0457f9f6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:07.776732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a3356f-bd3b-4362-9ada-2b0457f9f6d8 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:07.778105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 733/1473] 10.222.0.45 () {54 vars in 962 bytes} [Tue May 24 15:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:07.783439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9337-f4ca-4e84-896d-dc221b4dd39c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:07.783727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9337-f4ca-4e84-896d-dc221b4dd39c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:07.784003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be9337-f4ca-4e84-896d-dc221b4dd39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:07.801580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-58be9337-f4ca-4e84-896d-dc221b4dd39c None None] MFA Rules not processed for user `847292d322f6483ba5ff02294ad409dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:07.802751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-58be9337-f4ca-4e84-896d-dc221b4dd39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:07.809106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 741/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Tue May 24 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:07.816078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3356f-bd3b-4362-9ada-2b0457f9f6d8 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:07.826614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a3356f-bd3b-4362-9ada-2b0457f9f6d8 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:07.830577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:07.831198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:07.831478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:07.831746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:07.840596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:07.842032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f7e1c-6210-483e-9c77-1e8a7ac4dd56 service glance] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:07.851745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 734/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:14:10.112982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75437ef2-cbab-4120-98eb-8a6267394891 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:10.113466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75437ef2-cbab-4120-98eb-8a6267394891 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:10.113572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75437ef2-cbab-4120-98eb-8a6267394891 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:10.114941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 742/1476] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:10.120010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c216c8-c028-401c-b73a-c1f1c50402de None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:10.120424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c216c8-c028-401c-b73a-c1f1c50402de None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:10.120586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c216c8-c028-401c-b73a-c1f1c50402de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:10.138308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b0c216c8-c028-401c-b73a-c1f1c50402de None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:10.139481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b0c216c8-c028-401c-b73a-c1f1c50402de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:10.147173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 735/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:10.169752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75437ef2-cbab-4120-98eb-8a6267394891 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:10.182463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75437ef2-cbab-4120-98eb-8a6267394891 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:10.186854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:10.188097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:10.188406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:10.188686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:10.189419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:10.191181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:14:10.191181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:14:10.191181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:14:10.192382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055478a-ecba-4a22-98cf-62f209c97635 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:10.211877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 743/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Tue May 24 15:14:10 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:14:12.403143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f3e2-64cf-4910-b193-1472c2a6350a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:12.403657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f3e2-64cf-4910-b193-1472c2a6350a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:12.403657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5f3e2-64cf-4910-b193-1472c2a6350a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:12.405046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 736/1479] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:12.410930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061070c-a887-4f8e-9cc3-9e601be4821c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:12.411377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061070c-a887-4f8e-9cc3-9e601be4821c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:12.411481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c061070c-a887-4f8e-9cc3-9e601be4821c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:12.430883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c061070c-a887-4f8e-9cc3-9e601be4821c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:12.432084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c061070c-a887-4f8e-9cc3-9e601be4821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:12.439903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 744/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:12.484086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5f3e2-64cf-4910-b193-1472c2a6350a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:12.498470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5f3e2-64cf-4910-b193-1472c2a6350a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:12.504351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:12.504983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:12.505267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:12.505545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:12.513603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:12.515260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22dc22-7e1d-41b2-82cf-93b02c207e91 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:12.525105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 737/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:14:14.876723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94308e-6479-46d8-851f-74889d494413 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:14.877287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94308e-6479-46d8-851f-74889d494413 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:14.877287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94308e-6479-46d8-851f-74889d494413 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:14.878665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 745/1482] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:14.883981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae08-7c50-4393-92ed-604c4a8c94f2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:14.884394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae08-7c50-4393-92ed-604c4a8c94f2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:14.884583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579ae08-7c50-4393-92ed-604c4a8c94f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:14.902393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f579ae08-7c50-4393-92ed-604c4a8c94f2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:14.903578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f579ae08-7c50-4393-92ed-604c4a8c94f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:14.911320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 738/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:14.946880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94308e-6479-46d8-851f-74889d494413 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:14.958088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94308e-6479-46d8-851f-74889d494413 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:14.961615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:14.962246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:14.962527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:14.962799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:14.971477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:14.972946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fe0c3-7387-49c9-a490-b11f517096dd service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:14.982570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 746/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:14:17.194119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18c77-6d75-4090-b2eb-e5754787dd83 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:17.194623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18c77-6d75-4090-b2eb-e5754787dd83 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:17.194623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f18c77-6d75-4090-b2eb-e5754787dd83 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:17.196027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 739/1485] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:17.201413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3324a1-3ce3-4a5e-9093-86b9ccc5230c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:17.201830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3324a1-3ce3-4a5e-9093-86b9ccc5230c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:17.201978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3324a1-3ce3-4a5e-9093-86b9ccc5230c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:17.219909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ac3324a1-3ce3-4a5e-9093-86b9ccc5230c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:17.221129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ac3324a1-3ce3-4a5e-9093-86b9ccc5230c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:17.229112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 747/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:17.267485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f18c77-6d75-4090-b2eb-e5754787dd83 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:17.277952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f18c77-6d75-4090-b2eb-e5754787dd83 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:17.281439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:17.282064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:17.282344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:17.282617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:17.291308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:17.292818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12713567-0757-4292-b0c8-8e9db271ebd9 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:17.301989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 740/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:14:19.589231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6d883-f3d5-4ee8-8be6-42754dbac9ad None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:19.589817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6d883-f3d5-4ee8-8be6-42754dbac9ad None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:19.589817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6d883-f3d5-4ee8-8be6-42754dbac9ad None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:19.591145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 748/1488] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:19.596260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076030-2ce8-4f54-90db-797be08669c7 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:19.596722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076030-2ce8-4f54-90db-797be08669c7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:19.596839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29076030-2ce8-4f54-90db-797be08669c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:19.614722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-29076030-2ce8-4f54-90db-797be08669c7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:19.615891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-29076030-2ce8-4f54-90db-797be08669c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:19.623467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 741/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:14:19.662312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6d883-f3d5-4ee8-8be6-42754dbac9ad None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:19.673350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6d883-f3d5-4ee8-8be6-42754dbac9ad None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:19.676770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:19.677395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:19.677674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:19.677945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:19.686705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:19.688778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cf057-4502-4f6b-a678-c28b4830df5f service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:19.697924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 749/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:14:22.172531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892614-8cc9-4e06-91ee-fbedf4eacfb9 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:22.173113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892614-8cc9-4e06-91ee-fbedf4eacfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:22.173113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30892614-8cc9-4e06-91ee-fbedf4eacfb9 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:22.174469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 742/1491] 10.222.0.45 () {54 vars in 926 bytes} [Tue May 24 15:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:14:22.179668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a28496-996b-40fb-8767-d48568965451 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:22.180095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a28496-996b-40fb-8767-d48568965451 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:22.180242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a28496-996b-40fb-8767-d48568965451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:22.197989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e0a28496-996b-40fb-8767-d48568965451 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:14:22.199161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e0a28496-996b-40fb-8767-d48568965451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:14:22.206861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 750/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Tue May 24 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:14:22.244637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30892614-8cc9-4e06-91ee-fbedf4eacfb9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:14:22.255676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30892614-8cc9-4e06-91ee-fbedf4eacfb9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:14:22.259137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:14:22.259762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:14:22.260038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:14:22.260325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:14:22.268914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:14:22.270355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0e66c-85af-4fdc-9bdd-ecd1c8da2f81 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:14:22.279521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 743/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:14:32.192970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=33 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:14:32.275499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=33 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:16:08.886407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:08.886999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:08.886999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:08.905228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:16:08.906417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:08.914403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 751/1494] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:16:08.924075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:08.924579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:08.924693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:08.941981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:16:08.943168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:08.950808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 744/1495] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:16:08.955227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:08.964731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65e7a5b-3653-4455-aac8-53517a2b14e3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:08.969699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:08.970341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:08.970615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:08.970885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:08.971596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:16:08.971899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:08.972952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:08.972952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:16:08.972952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:16:08.973819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:08.986047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 752/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:16:08.993940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.002888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416be53-ae35-4490-9391-26cb7ffd86e9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.007468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.008101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.008418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.008692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.009366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.011128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.011128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:16:09.011128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:16:09.012323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.048108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 745/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:16:08 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:16:09.052574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.063000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c302f-a514-4520-850b-d281d8c7f7db admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.119189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.119860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.120138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.120439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.121136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.122270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.122270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:16:09.122270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:16:09.123527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.143324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 753/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:16:09 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:16:09.147438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.158247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252b3dc-82d3-4981-ba7c-e4fbd1e251da admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.162774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.163428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.163711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.163987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.164742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.165799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.165799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:16:09.165799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:16:09.166689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.179181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 746/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:16:09.182954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.192552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f403d-3118-4c85-93f9-ccd32dfd9550 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.196941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.197584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.197862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.198133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.198844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.199848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.199848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:16:09.199848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:16:09.201232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadebba5-27c6-41a2-a9ba-7aad671adbef admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.214167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 754/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:16:09.218774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.228549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570356ea-05de-41ac-84c5-3bcb41388cb7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.233344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.234207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.234487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.234762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] PATH_INFO: `/v3/projects/cf54cc6ea36641fd8172e182d07ab2e4/users/0bf6e2018c364224a6750d83d242667e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.249019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf54cc6ea36641fd8172e182d07ab2e4, user_id=0bf6e2018c364224a6750d83d242667e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.250056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.250056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:16:09.250056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:16:09.252006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4c8a0a-97a0-43a5-b0c7-d9c91bc87bdb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.266775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 747/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:16:09 2022] PUT /identity/v3/projects/cf54cc6ea36641fd8172e182d07ab2e4/users/0bf6e2018c364224a6750d83d242667e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:16:09.277790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d23b3-f507-4608-926c-85f122c25245 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.278082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d23b3-f507-4608-926c-85f122c25245 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.278989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d23b3-f507-4608-926c-85f122c25245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.303931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9f3d23b3-f507-4608-926c-85f122c25245 None None] MFA Rules not processed for user `0bf6e2018c364224a6750d83d242667e`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:16:09.305133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9f3d23b3-f507-4608-926c-85f122c25245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:09.366774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 755/1502] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:16:09.379902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea038-54dc-4461-a68e-48f3a7c2f171 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.380191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea038-54dc-4461-a68e-48f3a7c2f171 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.380505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fea038-54dc-4461-a68e-48f3a7c2f171 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.381859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 748/1503] 10.222.0.45 () {54 vars in 963 bytes} [Tue May 24 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:16:09.386450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa49cc-37ee-4321-9b47-7aadec00d995 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.386736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa49cc-37ee-4321-9b47-7aadec00d995 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.387010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fa49cc-37ee-4321-9b47-7aadec00d995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.406193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a2fa49cc-37ee-4321-9b47-7aadec00d995 None None] MFA Rules not processed for user `94ce1868b4ac4d2a8273867aab1bfe80`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:16:09.407328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2fa49cc-37ee-4321-9b47-7aadec00d995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:09.451323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:16:09.459413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 756/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Tue May 24 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:16:09.467850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fea038-54dc-4461-a68e-48f3a7c2f171 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.478145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fea038-54dc-4461-a68e-48f3a7c2f171 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.481611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.482235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.482768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.483042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.491124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.492585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.501671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 749/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:16:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:16:09.542582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3b667-425d-427f-9fad-877f584a5e5b None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.542876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3b667-425d-427f-9fad-877f584a5e5b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.543153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b3b667-425d-427f-9fad-877f584a5e5b None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.544479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 757/1506] 10.222.0.45 () {52 vars in 829 bytes} [Tue May 24 15:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:16:09.547921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.610551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fd8be-c83d-4071-b4a1-6629436a6fe0 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.615160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.616029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.616334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.616616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] PATH_INFO: `/v3/users/0bf6e2018c364224a6750d83d242667e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.617861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf6e2018c364224a6750d83d242667e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.618892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:09.618892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:16:09.618892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:16:09.620109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.683675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf6e2018c364224a6750d83d242667e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:09.684391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf6e2018c364224a6750d83d242667e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:09.685536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf6e2018c364224a6750d83d242667e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:09.697714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf6e2018c364224a6750d83d242667e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:09.723077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf6e2018c364224a6750d83d242667e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:09.727105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:16:09.739929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 750/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:16:09 2022] DELETE /identity/v3/users/0bf6e2018c364224a6750d83d242667e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:16:09.744669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.744972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.745254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.765101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:16:09.766290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:09.824757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 758/1508] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:16:09.835707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:09.837279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:09.896701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9f3f9-e324-4a9c-9dfb-4ad73d9f545e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:09.900232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:09.900912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:09.901196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:09.901474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:09.910213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:09.911668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2221cca-f541-485f-8f85-3bab847bbc69 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:09.921371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 751/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:16:10.249040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:16:10.250574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:16:10.262535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdc867-c457-4a46-a9d2-8dd9ebca6da9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:16:10.268185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:16:10.269181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:16:10.269358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:16:10.269674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] PATH_INFO: `/v3/projects/cf54cc6ea36641fd8172e182d07ab2e4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:16:10.271565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf54cc6ea36641fd8172e182d07ab2e4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:16:10.272829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:16:10.272829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:16:10.272829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:16:10.274034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:16:10.333939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf54cc6ea36641fd8172e182d07ab2e4', 'request_id': 'req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:10.339858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf54cc6ea36641fd8172e182d07ab2e4', 'request_id': 'req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:10.350498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf54cc6ea36641fd8172e182d07ab2e4', 'request_id': 'req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:10.369813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] The token cache is being invalidate because project cf54cc6ea36641fd8172e182d07ab2e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:16:10.370115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:10.370813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ac858c2d-576a-4da9-89c2-355a0ea9eb3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:16:10.372419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 759/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:16:10 2022] DELETE /identity/v3/projects/cf54cc6ea36641fd8172e182d07ab2e4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:16:19.916841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:16:20.366826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:17:53.337578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:53.338132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:53.338132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:53.339415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:17:53.345003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212cae-3e25-426e-9848-6e33b57167b7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:53.345489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212cae-3e25-426e-9848-6e33b57167b7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:53.345589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09212cae-3e25-426e-9848-6e33b57167b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:53.363706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-09212cae-3e25-426e-9848-6e33b57167b7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:17:53.364952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-09212cae-3e25-426e-9848-6e33b57167b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:17:53.423309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:17:53.591080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:17:53.592777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:17:53.666294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b758a0-9dfb-4c0a-816a-1ac47dec62fc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:17:53.672214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:17:53.672879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:53.673162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:53.673437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:53.681474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:17:53.683077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638df0a-e1e1-477c-b3bd-c31fa9c76ed7 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:17:53.693040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 753/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:17:54.571783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e7a6-95ab-455f-a6bc-d8a2e878342a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:54.572376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e7a6-95ab-455f-a6bc-d8a2e878342a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:54.572376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e7a6-95ab-455f-a6bc-d8a2e878342a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:54.573765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:17:54.578470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5f33-47b1-4f6d-8cd8-e75acb0e96a7 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:54.578912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5f33-47b1-4f6d-8cd8-e75acb0e96a7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:54.579018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac5f33-47b1-4f6d-8cd8-e75acb0e96a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:54.597464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-4fac5f33-47b1-4f6d-8cd8-e75acb0e96a7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:17:54.598666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4fac5f33-47b1-4f6d-8cd8-e75acb0e96a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:17:54.606314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:17:54.724410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4363d-4547-493d-ae17-9c7e6c2e1e1c None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:54.724716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4363d-4547-493d-ae17-9c7e6c2e1e1c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:54.724983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4363d-4547-493d-ae17-9c7e6c2e1e1c None None] PATH_INFO: `/v3` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:54.726283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 762/1516] 10.222.0.48 () {54 vars in 925 bytes} [Tue May 24 15:17:54 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:17:54.733303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f648bd-ea66-4e5b-9f3d-8e1cdbe59d30 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:54.733592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f648bd-ea66-4e5b-9f3d-8e1cdbe59d30 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:54.733870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f648bd-ea66-4e5b-9f3d-8e1cdbe59d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:54.752081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f1f648bd-ea66-4e5b-9f3d-8e1cdbe59d30 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:17:54.753459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f1f648bd-ea66-4e5b-9f3d-8e1cdbe59d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:17:54.764685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4363d-4547-493d-ae17-9c7e6c2e1e1c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:17:54.779298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4363d-4547-493d-ae17-9c7e6c2e1e1c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:17:54.784884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:17:54.785524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:17:54.785808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:17:54.786084 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:17:54.795528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:17:54.797176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42911a48-82d5-4a7c-8312-5d1865b5fa74 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:17:54.806773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 763/1517] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:17:54.817088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 755/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Tue May 24 15:17:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:04.093525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f18-e284-4728-9c2d-78e69ba97df7 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:04.094038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f18-e284-4728-9c2d-78e69ba97df7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:04.094149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d1f18-e284-4728-9c2d-78e69ba97df7 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:04.095451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 764/1519] 10.222.0.45 () {54 vars in 960 bytes} [Tue May 24 15:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:04.100393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862388d-117c-490e-bf3e-f74d7b41bc8f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:04.100840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862388d-117c-490e-bf3e-f74d7b41bc8f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:04.100946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862388d-117c-490e-bf3e-f74d7b41bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:04.118810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3862388d-117c-490e-bf3e-f74d7b41bc8f None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:04.120019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3862388d-117c-490e-bf3e-f74d7b41bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:04.128541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 756/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Tue May 24 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:04.135909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d1f18-e284-4728-9c2d-78e69ba97df7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:04.149335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d1f18-e284-4728-9c2d-78e69ba97df7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:04.154756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:04.155373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:04.155651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:04.155922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:04.163906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:04.165506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7a2b5-604e-4566-81d2-a015c4d8c227 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:04.174908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 765/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:18:06.096023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc014d4-a6b4-4e03-94a1-030fd2b4e012 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:06.096530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc014d4-a6b4-4e03-94a1-030fd2b4e012 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:06.096641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc014d4-a6b4-4e03-94a1-030fd2b4e012 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:06.097974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:06.102799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5129589-a8c4-408b-910e-19979e86e06a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:06.103198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5129589-a8c4-408b-910e-19979e86e06a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:06.103384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5129589-a8c4-408b-910e-19979e86e06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:06.122616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f5129589-a8c4-408b-910e-19979e86e06a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:06.123851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f5129589-a8c4-408b-910e-19979e86e06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:06.131715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:06.290477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc014d4-a6b4-4e03-94a1-030fd2b4e012 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:06.305668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc014d4-a6b4-4e03-94a1-030fd2b4e012 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:06.311668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:06.312316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:06.312612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:06.312890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:06.321001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:06.322554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fc008-0220-4afd-a247-e7a2c72d7e57 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:06.331870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 758/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:07.187488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1dac9-7ef6-45bb-ad02-3873afe689b8 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:07.188035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1dac9-7ef6-45bb-ad02-3873afe689b8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:07.188035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c1dac9-7ef6-45bb-ad02-3873afe689b8 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:07.189557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 15:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:07.194202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ccda8-ec1e-4bdc-9004-72033415946f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:07.194600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ccda8-ec1e-4bdc-9004-72033415946f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:07.194760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ccda8-ec1e-4bdc-9004-72033415946f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:07.212419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ff2ccda8-ec1e-4bdc-9004-72033415946f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:07.213618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ff2ccda8-ec1e-4bdc-9004-72033415946f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:07.221848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:07.373769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1dac9-7ef6-45bb-ad02-3873afe689b8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:07.387892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c1dac9-7ef6-45bb-ad02-3873afe689b8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:07.393295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:07.393928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:07.394211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:07.394488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:07.402508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:07.404645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1d8e-b02e-4f7a-adc8-e2f713c96267 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:07.413913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 768/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:11.346553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:11.347064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:11.347064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:11.348460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:11.353801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62317-f346-48f6-86bc-b123a3ea6fbf None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:11.354170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62317-f346-48f6-86bc-b123a3ea6fbf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:11.354361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af62317-f346-48f6-86bc-b123a3ea6fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:11.373427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8af62317-f346-48f6-86bc-b123a3ea6fbf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:11.374608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8af62317-f346-48f6-86bc-b123a3ea6fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:11.382281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:11.418133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:11.420370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:11.474507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:18:11.478756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ead5-2817-4a50-b8b0-48c016ff0155 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:11.483247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:11.483882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:11.484160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:11.484466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:11.493262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:11.494702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b78656-1aee-4f0b-a3b9-36d523999835 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:11.504616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 761/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:13.660138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59abb-30c5-433e-b3c2-f14b57109a82 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:13.660734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59abb-30c5-433e-b3c2-f14b57109a82 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:13.660734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59abb-30c5-433e-b3c2-f14b57109a82 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:13.662148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:13.667239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84446c66-d705-42c5-bb30-ccc45088debf None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:13.667638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84446c66-d705-42c5-bb30-ccc45088debf None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:13.667829 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84446c66-d705-42c5-bb30-ccc45088debf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:13.685021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-84446c66-d705-42c5-bb30-ccc45088debf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:13.686200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-84446c66-d705-42c5-bb30-ccc45088debf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:13.693791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:13.729794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59abb-30c5-433e-b3c2-f14b57109a82 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:13.741099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59abb-30c5-433e-b3c2-f14b57109a82 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:13.744851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:13.745504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:13.745783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:13.746053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:13.756905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:13.758329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3f2f2-4231-42e5-ba58-f73446d5b36d service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:13.768165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 771/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:17.401792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:17.402373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:17.402373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:17.403724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:17.409200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82903-6edb-4c2a-bddc-9563f111bfb9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:17.409629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82903-6edb-4c2a-bddc-9563f111bfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:17.409740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d82903-6edb-4c2a-bddc-9563f111bfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:17.428006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-82d82903-6edb-4c2a-bddc-9563f111bfb9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:17.429228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-82d82903-6edb-4c2a-bddc-9563f111bfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:17.437574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:18:17.706185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:17.707849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:17.764714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52a169-fd79-4686-bb9b-b10e7dec9d1a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:17.768243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:17.768913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:17.769198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:17.769475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:17.778387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:17.780340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092dc41-876d-4cb7-beee-f66fb120cfb0 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:17.789647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 764/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:20.008493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:20.009100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:20.009100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:20.010429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 15:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:18:20.016176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c45f-fb86-4e46-85dd-d7bc6d927687 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:20.016612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c45f-fb86-4e46-85dd-d7bc6d927687 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:20.016769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c45f-fb86-4e46-85dd-d7bc6d927687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:20.035163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b282c45f-fb86-4e46-85dd-d7bc6d927687 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:18:20.036384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b282c45f-fb86-4e46-85dd-d7bc6d927687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:20.044077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 15:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:18:20.316434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:18:20.318150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:18:20.331852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28658a16-af29-4d37-aa11-53831e82d59a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:18:20.335325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:18:20.335955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:18:20.336233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:18:20.336535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:18:20.345936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:18:20.347345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f00866-d284-49d8-8324-a07ea3f1154a service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:18:20.356627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 774/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:18:30.030405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=36 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:18:30.352596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=66 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:50.287382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe087-abe1-484e-b75c-399c3d055c06 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.288524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe087-abe1-484e-b75c-399c3d055c06 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.289047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffe087-abe1-484e-b75c-399c3d055c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.320539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8d00-2af9-4485-bef0-7aef3216da41 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.321645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8d00-2af9-4485-bef0-7aef3216da41 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.322171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f8d00-2af9-4485-bef0-7aef3216da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.326121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-45ffe087-abe1-484e-b75c-399c3d055c06 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.327348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-45ffe087-abe1-484e-b75c-399c3d055c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.346063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 766/1540] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.352765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f42f8d00-2af9-4485-bef0-7aef3216da41 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.353722 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f42f8d00-2af9-4485-bef0-7aef3216da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.354231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.354529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.354802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.368032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 775/1541] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.370794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12437b-2a2c-4a43-a6d7-27bee57ab5f4 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.371865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12437b-2a2c-4a43-a6d7-27bee57ab5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.376413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12437b-2a2c-4a43-a6d7-27bee57ab5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.403795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2f12437b-2a2c-4a43-a6d7-27bee57ab5f4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.405300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2f12437b-2a2c-4a43-a6d7-27bee57ab5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.414670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.414670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.417236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 776/1542] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.419487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428a21-2190-4afb-9baf-647edeaf1dc5 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.420516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428a21-2190-4afb-9baf-647edeaf1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.421474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26428a21-2190-4afb-9baf-647edeaf1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.435788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 767/1543] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:19:50.436528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.454397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-26428a21-2190-4afb-9baf-647edeaf1dc5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.456008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-26428a21-2190-4afb-9baf-647edeaf1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.466971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1cb81-8dfd-46ea-9aac-39222f84da9c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.468385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 777/1544] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.470047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.470517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.470954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.483527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.484209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.484529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.484803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.485736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:50.486216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.487124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:50.487124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:19:50.487124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:19:50.488060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bffc8-b5c5-4403-b0ea-eabafeff3187 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.505532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.506906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.518080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 778/1545] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.519041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.526928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 768/1546] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:50.527985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ef48-f972-4439-aace-88687a8a0add None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.528276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ef48-f972-4439-aace-88687a8a0add None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.528583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203ef48-f972-4439-aace-88687a8a0add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.536479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb668c3-9d84-4f42-8408-e4bdc4b4cdad None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.543475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.544149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.544466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.544741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.545820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:50.546263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.548229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec9b03-070e-43a2-9049-4290df0a3306 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.566144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 779/1547] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:50.567732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.568023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.568319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.573272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f203ef48-f972-4439-aace-88687a8a0add None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.574907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f203ef48-f972-4439-aace-88687a8a0add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.583309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 769/1548] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.584140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203ef48-f972-4439-aace-88687a8a0add None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.589550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.591476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.594482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203ef48-f972-4439-aace-88687a8a0add None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.599530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.600197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.600508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.600906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.601642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:50.601948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.603566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40709832-c687-4d96-81e7-624e99e2267d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.606433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 780/1549] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.607772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.618177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4b359-cdcd-463f-a52d-b0577a1a95ac None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.623782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.624494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.624778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.625050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.625748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.626782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:50.626782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:19:50.626782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:19:50.627608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 770/1550] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:50.628377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.632933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394382b-1244-4641-bfb8-d2dc7d61d242 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.633243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394382b-1244-4641-bfb8-d2dc7d61d242 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.633507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394382b-1244-4641-bfb8-d2dc7d61d242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.656286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2394382b-1244-4641-bfb8-d2dc7d61d242 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.657504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2394382b-1244-4641-bfb8-d2dc7d61d242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.661387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 781/1551] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:50.662199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.665533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 771/1552] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.666751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b263d6-8334-415b-9a8d-ce17c9138b6e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.667031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b263d6-8334-415b-9a8d-ce17c9138b6e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.667299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b263d6-8334-415b-9a8d-ce17c9138b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.685060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-39b263d6-8334-415b-9a8d-ce17c9138b6e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.686273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-39b263d6-8334-415b-9a8d-ce17c9138b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.726074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb438e5-9bf6-4740-a9b4-c609a9876650 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.731270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.731930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.732214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.732534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.733490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.735384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f3839e-e860-422e-8aa7-94f48e4e6d04 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.744537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 772/1553] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.745913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd630-bc5f-4a0d-94f2-15fb3b5b5b53 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.746272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd630-bc5f-4a0d-94f2-15fb3b5b5b53 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.746624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd630-bc5f-4a0d-94f2-15fb3b5b5b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.762309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 782/1554] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/projects => generated 403 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:50.763930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.764226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.764530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.765474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-004fd630-bc5f-4a0d-94f2-15fb3b5b5b53 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.766511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-004fd630-bc5f-4a0d-94f2-15fb3b5b5b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.783038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.784416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.824779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 773/1555] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.825985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.826268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.826545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.845047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.845517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 783/1556] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.846355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.846479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.855194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 774/1557] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.856657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.857079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13640dc2-8df5-4364-9810-56072fd77cfb None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.866650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aa089-a610-4b7f-8db3-b419caaf3d25 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.868140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.868841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.869117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.869395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.870111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.871234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.871878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.872145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.872256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.872459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.873138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.874163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:50.874163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:19:50.874163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:19:50.875367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535a748-737c-4d5d-9b07-de277991f390 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.887389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:50.896522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 775/1558] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:50.897894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.898201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.898528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.902648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 784/1559] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:50.903339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.919197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:50.920568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:50.939108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:50.969482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae1d6f-ea4e-4f79-84a2-6b9536eb2da1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.974627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.975561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.975922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.976271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.977223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:50.977604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:50.979279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd694-d349-4e11-97e0-1f3ca14945dd admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:50.982729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 776/1560] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:50.983811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:50.992380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 785/1561] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:50.993717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da420e6-76fe-4f80-9d77-d50349f8a150 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:50.993844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7662-e6bb-4511-9779-a9e76897f091 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.994020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7662-e6bb-4511-9779-a9e76897f091 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.994346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a7662-e6bb-4511-9779-a9e76897f091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:50.998286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:50.998958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:50.999467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:50.999750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.000459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.002417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.019502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5d8a7662-e6bb-4511-9779-a9e76897f091 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:51.020737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5d8a7662-e6bb-4511-9779-a9e76897f091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:51.029164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 777/1562] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/users => generated 302 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.029861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 786/1563] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:51.030961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.031250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.031525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.033793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.044966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1fdc08-193c-48cf-83da-e22457a87ae2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.048899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:51.049438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.050074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:51.050193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.050360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.050636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.051690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:51.051999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.053633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.058020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 787/1564] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:51.058901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.066800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 778/1565] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:51.069221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.071851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36eded-0a9f-4d43-a752-712283a654aa None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.077503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.078284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.078603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.078983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.079947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.080321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862531db-fccb-486c-93f2-30519a7a6dac admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.081811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.085437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.086100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.086384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.086656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.087803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.090376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.092923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 788/1566] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.093716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.103338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89dad3d-b94c-42cf-b50f-d26747c0731e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.109141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.109841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.110118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.110391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.111140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:51.111441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.111809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 779/1567] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.112782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.113131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.122473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d717-84f3-4e58-8103-8b127a75f832 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.131137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 789/1568] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:51.131137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.131360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.131360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.131360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.131360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.131679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.131679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.140601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6625b30-5d0f-4d7f-b0b2-0af017b559a9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.145236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.145900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.146179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.146450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.147278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.148809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.158386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 780/1569] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.162296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.162422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 790/1570] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.162422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.172017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b7b69-7e13-4ea6-b874-47dedd0a5da2 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.224753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200174-e693-456c-bcde-9df4140f2c9f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.228545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.229210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.229419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.229544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.229751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.230074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.230410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.230521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:51.230688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.230790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.231405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:19:51.231713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.232389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.233662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.244525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 791/1571] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:51.251765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 781/1572] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:19:51.251868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.251969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.256818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46217b-81ce-4b1c-87ad-cf68e5830480 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.261016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7bb85-dd64-424e-a374-85fc55839328 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.262239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.262891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.263176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.263456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.264140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.266088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.266218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.266731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.267026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.267296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.268043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.269682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.280503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 782/1573] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.281287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.292353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 792/1574] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.292955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db99b0-bf54-4e4b-bea9-2321db7c7b9b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.293457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.307608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.328007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.352663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.353367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.353652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.353958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.354649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.356915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.359423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c845d-944b-495f-8845-569a9fe6e0fa admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.363963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.364660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.364945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.365221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.365905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.366809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 783/1575] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.370057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.370156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.379746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714ca8-97d0-483c-b504-11d9a1609432 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.384286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.384959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.385360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.385641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.386361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.388342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.395259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 793/1576] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.396166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.412972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 784/1577] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.412972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.421149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8be31e-a63e-4f08-83e6-552788bfb72b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.458632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcda81-247f-4db8-8fbc-f889a327a049 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.463201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.463861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.464146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.464450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.465654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.467238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.476843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 794/1578] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.477288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.477599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.477952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.478290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.478498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.479183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.481755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.488009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8349a7a-eaa5-4bf7-a94c-6a63793ad116 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.492690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.493324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.493606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.493881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.494587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.496562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.507738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 785/1579] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.507738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.522902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 795/1580] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.523768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.571547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a984cf97-af82-4c24-860b-d1608a9c7899 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.584554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e66b96-61c6-404c-b8bb-d5e4027f8993 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.589062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.590256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.590538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.590814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.591522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.593465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.615296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 796/1581] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.616099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.618812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.620825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.626074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53620bc7-f3f4-419c-8e71-ca7a8e9cc3af admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.630222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.631118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.631401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.631638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.631771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] PATH_INFO: `/v3/projects/cb5a1867355149cd844bf31c563b9866/users/8120c09231af4074a4d52db426e4e4cc/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.632279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.632589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.632869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.633589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.635174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.644962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 797/1582] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.645725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.646425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5a1867355149cd844bf31c563b9866, user_id=8120c09231af4074a4d52db426e4e4cc, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.649441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.657596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05187e62-1fa0-4cfc-826d-026945d9a18c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.662561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 786/1583] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:51 2022] PUT /identity/v3/projects/cb5a1867355149cd844bf31c563b9866/users/8120c09231af4074a4d52db426e4e4cc/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:51.663442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.715335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.716002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.716281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.716580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.717264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.719169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.726281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3b7ff-8c0e-4a29-9ef8-7a29560318e6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.732076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.732830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.733385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.733695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.734634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.736257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.739465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 798/1584] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.740238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.747824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 787/1585] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.747824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.751112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafca14-fdc9-4245-ac1d-5b93fc62b0b6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.755719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.756614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.756892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.757164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] PATH_INFO: `/v3/projects/1767066f43cc482bba115ccb2e1c1b1e/users/1bd28cbcb7c4484cb4c52710eb4b660d/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.757275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f0009-fa97-426f-9335-b2e0fb0bc197 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.762191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.762827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.763114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.763390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.764064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.766298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.772132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1767066f43cc482bba115ccb2e1c1b1e, user_id=1bd28cbcb7c4484cb4c52710eb4b660d, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.773185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:51.773185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:19:51.773185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:19:51.775078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.786530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 788/1586] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 289 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.787328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.788547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 799/1587] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:51 2022] PUT /identity/v3/projects/1767066f43cc482bba115ccb2e1c1b1e/users/1bd28cbcb7c4484cb4c52710eb4b660d/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:51.796937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849919a6-31a3-4ef4-a8bb-b176533d4b71 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.797340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.854301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.854930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.855237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.855509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.856217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.858462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.859621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d0902-c05d-4c7e-ab8c-63ffc8a7a442 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.864160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.864880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.865149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.865422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.866150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.868268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57f2c7-1734-4f9a-ae98-a0ce3b435168 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.878579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 789/1588] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:51.879043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 800/1589] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:51.880128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.880457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.880738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.881560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.893284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c93798-1983-4b64-b882-0da8c076aec8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.897805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.898785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:51.898952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:51.899227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] PATH_INFO: `/v3/projects/0812b13a287c43259d7e09b0c658d4c8/users/2befe74ad06542b785d326101fc06d0b/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:51.907653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] MFA Rules not processed for user `8120c09231af4074a4d52db426e4e4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:51.908900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:51.914600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0812b13a287c43259d7e09b0c658d4c8, user_id=2befe74ad06542b785d326101fc06d0b, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:51.917347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:51.931561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 790/1590] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:51 2022] PUT /identity/v3/projects/0812b13a287c43259d7e09b0c658d4c8/users/2befe74ad06542b785d326101fc06d0b/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:51.932499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.952137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.952576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:51.978368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 801/1591] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:51.979393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:51.993318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683cb345-d206-4918-929f-4b9418a9a031 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:51.998856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:51.999841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.000218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.000626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.001503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.002737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd220cbb-1553-49c5-b0f3-3c4c9de36416 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.003289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.007518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.008331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.008612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.008923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.009668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.011420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf83df0-f116-46c9-8ecf-2e9f15f841d4 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.018903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 802/1592] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.018903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.028103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 791/1593] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.028103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bb060-b419-484e-abde-fe4480659091 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.028103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bb060-b419-484e-abde-fe4480659091 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.028103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bb060-b419-484e-abde-fe4480659091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.028591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8656867c-ecbe-481d-ad26-0d72044052a6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.032867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.033521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.033802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.034080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.034795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.036410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.046754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 803/1594] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.047538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.056476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9b5bb060-b419-484e-abde-fe4480659091 None None] MFA Rules not processed for user `1bd28cbcb7c4484cb4c52710eb4b660d`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.057020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605a686-0400-4be3-9fb4-be397c2cf0c5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.057660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b5bb060-b419-484e-abde-fe4480659091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.062137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.062728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.063008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.063284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.063989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.065597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.076530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 804/1595] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.077366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.089039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdefc059-ad03-4d9b-9d2d-72877aaf2cfb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.094004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.094640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.094919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.095192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.095941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.097586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378288a6-fb52-4213-ad00-e783a368e80f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.108249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 805/1596] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.109419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.109712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.109987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.127131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.128346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.129270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 792/1597] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.130598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c817-cf9e-4f83-8ab7-de5aa02c3a03 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.130864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c817-cf9e-4f83-8ab7-de5aa02c3a03 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.131138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2c817-cf9e-4f83-8ab7-de5aa02c3a03 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.132460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 793/1598] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 15:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:19:52.133145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c817-cf9e-4f83-8ab7-de5aa02c3a03 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.136423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 806/1599] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.137727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.147717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cf1e3-e04f-4372-8ce5-7de61eb13f3b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.148810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2c817-cf9e-4f83-8ab7-de5aa02c3a03 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.152836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.153491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.153770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.153929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.154051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.154752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.154857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.155071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.155347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] PATH_INFO: `/v3/projects/e8d86717f1954ab7a59cdef0cf13556c/users/2d8a7414fd254ec981d35b417dc373cd/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.156365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.166473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 807/1600] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.167375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.171230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d86717f1954ab7a59cdef0cf13556c, user_id=2d8a7414fd254ec981d35b417dc373cd, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.174001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.177526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e7368-d2af-4447-8950-762f0fb9551c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.182196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.182913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.183190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.183490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.184200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.185794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.187249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 794/1601] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:52 2022] PUT /identity/v3/projects/e8d86717f1954ab7a59cdef0cf13556c/users/2d8a7414fd254ec981d35b417dc373cd/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:52.190784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.202204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 808/1602] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.202204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.210147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47393c6-0368-4be9-b9ea-f2c8cb94056e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.251142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f3a84-ea98-4ade-b21c-c4da52f97a21 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.255816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.256635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.256912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.257194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] PATH_INFO: `/v3/projects/0fe1d758c40542e9b237f9d7bcfa5755/users/40db5795ce7e4c5bbf06fb81ce404e78/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.271852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe1d758c40542e9b237f9d7bcfa5755, user_id=40db5795ce7e4c5bbf06fb81ce404e78, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.272267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.272992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.273273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.273542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.274256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.274546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3655ca-a58c-443b-a9b3-73c5bf5fafb5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.275847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.283851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:52.286247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 809/1603] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:52.287298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.292010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 795/1604] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:52 2022] PUT /identity/v3/projects/0fe1d758c40542e9b237f9d7bcfa5755/users/40db5795ce7e4c5bbf06fb81ce404e78/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:52.293531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.294422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.294701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.299301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50747eb7-5c69-4c62-b0f7-43b73f666c4c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.312495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.313678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.356947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.357615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.358147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.358509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.359144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.361091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.371986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:52.383070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 796/1605] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.384138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.392912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 810/1606] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:52.393732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.439424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109f9c8-0de8-4977-a50d-178a8c958369 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.468666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a9104-8851-4ab6-b328-56a7b7475641 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.473328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.474225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.474502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.474785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] PATH_INFO: `/v3/projects/97174fe5a2f44eb48a6bbb17756601c6/users/83b2f2db9bd24ab9954693480d81df44/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.489763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97174fe5a2f44eb48a6bbb17756601c6, user_id=83b2f2db9bd24ab9954693480d81df44, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.491487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.492143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.492445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723176e0-c034-46e3-852f-2fd234806386 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.492579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.492794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.503102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.504686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.505869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 811/1607] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:52 2022] PUT /identity/v3/projects/97174fe5a2f44eb48a6bbb17756601c6/users/83b2f2db9bd24ab9954693480d81df44/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:52.507209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.507500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.507781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.514392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 797/1608] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:52.517227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.528189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460acecc-9215-4982-a86f-eeba4eefcdf2 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.530915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.532146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.591175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:52.597763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 812/1609] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.602258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.606438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.607352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.607639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.608171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] PATH_INFO: `/v3/projects/b175a657eab24493b2adbec405cbd515/users/0dc17670edbb4cc797103b0d41835bf9/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.614335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1e8d95-038c-45d3-a4cf-e1d5b408d2cf None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.619551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.620563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.620906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.621695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] PATH_INFO: `/v3/projects/e3d7790a5da14f9090ecbfc53c3f7b49/users/0905caa551494b479b47da44cc0b7de7/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.626725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b175a657eab24493b2adbec405cbd515, user_id=0dc17670edbb4cc797103b0d41835bf9, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.629562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff94989-410e-4802-a698-7acea5f67273 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.637848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d7790a5da14f9090ecbfc53c3f7b49, user_id=0905caa551494b479b47da44cc0b7de7, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.640676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.645896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 798/1610] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:52 2022] PUT /identity/v3/projects/b175a657eab24493b2adbec405cbd515/users/0dc17670edbb4cc797103b0d41835bf9/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:52.647544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.647883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.648157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.663713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 813/1611] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:52 2022] PUT /identity/v3/projects/e3d7790a5da14f9090ecbfc53c3f7b49/users/0905caa551494b479b47da44cc0b7de7/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:52.663713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.676126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.677589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.729199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:52.747138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 799/1612] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.748450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.760262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd6443-31fd-49da-9f49-d561dbaa2f4c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.763315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebfa65-8806-4115-a004-81dfbb9bc080 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.765647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.766328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.766611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.766887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.767583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.769463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.769612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd8c93e-f546-4d95-a9a4-b01f1f9be71c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.770115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.770396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.770675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.797183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 800/1613] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/users => generated 309 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:52.797183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae513e32-7da9-4523-a183-f4484d4403e9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.797183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae513e32-7da9-4523-a183-f4484d4403e9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.797183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae513e32-7da9-4523-a183-f4484d4403e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.827472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ae513e32-7da9-4523-a183-f4484d4403e9 None None] MFA Rules not processed for user `83b2f2db9bd24ab9954693480d81df44`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.828718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ae513e32-7da9-4523-a183-f4484d4403e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.835823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.837451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.847720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 814/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:52.853224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.854864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.919272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 801/1615] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.921292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baae39c-b549-4ebc-a6e4-627a2e33951e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.921822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baae39c-b549-4ebc-a6e4-627a2e33951e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.922304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baae39c-b549-4ebc-a6e4-627a2e33951e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.939586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176bd10-860b-4cc3-bcf8-84135227030d service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.943663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.943961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-4baae39c-b549-4ebc-a6e4-627a2e33951e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:52.944358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.944645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.944918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.946271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4baae39c-b549-4ebc-a6e4-627a2e33951e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:52.955946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 802/1616] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:52.956540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:52.958206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:52.960369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.960889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.961368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:52.972223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 815/1617] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:52.973126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:52.983918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ebd660-7401-4edc-89c0-5597551f48e0 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:52.989496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:52.990341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:52.990625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:52.990900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.002570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.004081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] MFA Rules not processed for user `0905caa551494b479b47da44cc0b7de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:53.004232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.010067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:53.015660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 816/1618] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:53.016562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.030146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e915a-41f0-4535-b913-4006e9fcdd08 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.036043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.037238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.037519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.037793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.038525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.040594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.051700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:53.056072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 817/1619] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:53.057289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.072480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de10ec-0fde-4feb-b6d6-71e5325780e5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.079368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.080089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.080413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.080689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.081900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.085186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.111951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 803/1620] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:53.113114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.127470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923898a-4aa7-4a92-9d1c-dba976ed0a8b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.131842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.132516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.132898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.133266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.143876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.145790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.157782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 804/1621] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:53.158740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.163826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 818/1622] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:19:53 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:53.164846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.170279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e91f3-991e-4302-8e48-bef7d50f9184 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.233983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.235024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.235424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.235928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] PATH_INFO: `/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b/users/f901441f65c84742b3007edd40a2b135/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.238090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:53.252597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e99d527c5247bcb2ea4e9fdf7feb8b, user_id=f901441f65c84742b3007edd40a2b135, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.255352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821f53d-bb50-491a-be26-615636d9a275 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.262242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a988c-8513-4843-bcd4-0816666e79f0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.269075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.270249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.270475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.270735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.271774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.274200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed882fa0-946b-4283-97c3-31bc61d50239 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.275642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 805/1623] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:53 2022] PUT /identity/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b/users/f901441f65c84742b3007edd40a2b135/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:53.277823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df908a0-4044-449c-ab72-338f310c6959 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.278127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df908a0-4044-449c-ab72-338f310c6959 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.278404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df908a0-4044-449c-ab72-338f310c6959 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.280288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 806/1624] 10.222.0.45 () {54 vars in 961 bytes} [Tue May 24 15:19:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:19:53.282015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c9fbd-406b-4005-9552-fd8f50829c24 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.282156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c9fbd-406b-4005-9552-fd8f50829c24 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.282500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c9fbd-406b-4005-9552-fd8f50829c24 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.284368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 807/1625] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:19:53.285188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c9fbd-406b-4005-9552-fd8f50829c24 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.312906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 819/1626] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:19:53 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:19:53.314657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.315042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.315433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.340785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:53.342032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:53.378450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c9fbd-406b-4005-9552-fd8f50829c24 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.383977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.385710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.386089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.387597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.388528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.391207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec6efc-87a0-49d8-af7f-ed65bda0effb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.410613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 808/1627] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:53.410613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.411043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.411618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.421787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 820/1628] 10.222.0.45 () {58 vars in 1058 bytes} [Tue May 24 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:53.422919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.436023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853160cb-5029-47d3-b179-fae6e9398845 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.440389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:53.441369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.442062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.442826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.442954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.443714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.444086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:53.445669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.457083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 809/1629] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:53.458290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.458656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 821/1630] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:53.460083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.473252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314e33f-6725-49ed-9105-faae0ad42140 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.479163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.480893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.481284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.481697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] PATH_INFO: `/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b/users/f901441f65c84742b3007edd40a2b135/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.487915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec98ecd6-ff33-49db-b41b-985d80225430 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.497509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.498768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.499028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62e99d527c5247bcb2ea4e9fdf7feb8b, user_id=f901441f65c84742b3007edd40a2b135, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.499609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.500362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.505781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.527100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 810/1631] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:53 2022] PUT /identity/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b/users/f901441f65c84742b3007edd40a2b135/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:53.528451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.552893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:53.598077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.601028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.614974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 822/1632] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:53.624014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dbd0f-d2af-4fcf-8653-a7506c6ea122 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.626585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.632784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.633959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.634722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.635331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] PATH_INFO: `/v3/projects/439d0ca051354554bc86127f029ebddd/users/7300fbec15694ab89839101334ed1522/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.662361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=439d0ca051354554bc86127f029ebddd, user_id=7300fbec15694ab89839101334ed1522, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.666460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc09fb80-9880-426b-963a-73f56ef689f1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:53.702210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 811/1633] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:53 2022] PUT /identity/v3/projects/439d0ca051354554bc86127f029ebddd/users/7300fbec15694ab89839101334ed1522/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:53.704348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.704677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.705036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.748681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02667968-19c2-4315-9610-b4f166d0afaf service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:53.757953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:53.764010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] MFA Rules not processed for user `f901441f65c84742b3007edd40a2b135`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:53.765768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:53.852162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:53.852454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:53.852682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:53.853454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:53.930488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 812/1634] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:53.932231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:53.997572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:53.999067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.028641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 823/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:54.029606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:54.050973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee544aef-73c4-47b1-ab17-a67a18f1ac78 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:54.054682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a4ba92-5349-47f1-b17c-8b288a0c572d service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:54.059353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:54.061127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.063787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.064624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.067782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:54.069496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.077930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:54.078602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.078883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.079157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.096029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 813/1636] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:19:54.100248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:54.106558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:54.108278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ef7b0-0a3a-4c0c-99ba-7a899a017180 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.127604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfa7ae-9f75-40a4-8556-06d10eb63266 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:54.134562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:54.135612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.135900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.136190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] PATH_INFO: `/v3/projects/439d0ca051354554bc86127f029ebddd/users/7300fbec15694ab89839101334ed1522/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.144116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] RBAC: Authorizing `identity:create_grant(project_id=439d0ca051354554bc86127f029ebddd, user_id=7300fbec15694ab89839101334ed1522, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:54.145418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 824/1637] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:54.149643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3e6d7-8fdd-4300-93de-a98660ff02db admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.175508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 814/1638] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:19:54 2022] PUT /identity/v3/projects/439d0ca051354554bc86127f029ebddd/users/7300fbec15694ab89839101334ed1522/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:54.188016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.188647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.189210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.197974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2288b-d5fb-4131-b3ba-72524a0aafc9 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.198297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2288b-d5fb-4131-b3ba-72524a0aafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.198543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2288b-d5fb-4131-b3ba-72524a0aafc9 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.202540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 815/1639] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 15:19:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:19:54.212629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.212950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.213189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.232112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] MFA Rules not processed for user `7300fbec15694ab89839101334ed1522`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:54.234727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:54.249528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:54.257779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:54.260594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:54.404388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 816/1640] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:54.412178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 825/1641] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:54.470569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:54.487174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:54.566193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:54.624405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ce657-1c17-41a8-bdc8-da2bd90bc286 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:54.647743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177763d-594a-4c9c-beb2-4bdb274011bc None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:54.650625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:54.651672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:54.652603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.652885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.653087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.653197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.653432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.653561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] PATH_INFO: `/v3/users/1bd28cbcb7c4484cb4c52710eb4b660d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.656326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd28cbcb7c4484cb4c52710eb4b660d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:54.663846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.682172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:54.683977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ea7553-ef09-4348-8652-3c26af2700b9 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:54.717735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 826/1642] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:54.808883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd28cbcb7c4484cb4c52710eb4b660d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:54.811094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd28cbcb7c4484cb4c52710eb4b660d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:54.811982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd28cbcb7c4484cb4c52710eb4b660d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:54.835470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd28cbcb7c4484cb4c52710eb4b660d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:54.876225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd28cbcb7c4484cb4c52710eb4b660d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:54.904260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 817/1643] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:19:54 2022] DELETE /identity/v3/users/1bd28cbcb7c4484cb4c52710eb4b660d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:54.908538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcf69-3fd9-419c-af6f-f36fcef67bc6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:54.909574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcf69-3fd9-419c-af6f-f36fcef67bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:54.909574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbcf69-3fd9-419c-af6f-f36fcef67bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:54.967518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0fcbcf69-3fd9-419c-af6f-f36fcef67bc6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:54.974674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0fcbcf69-3fd9-419c-af6f-f36fcef67bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.125132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 827/1644] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:55.148547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:55.150034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.163952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1fa0f-330e-46d5-ae54-9b1fc8837b56 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:55.164382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1fa0f-330e-46d5-ae54-9b1fc8837b56 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:55.164702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1fa0f-330e-46d5-ae54-9b1fc8837b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:55.207281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:55.241463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-03a1fa0f-330e-46d5-ae54-9b1fc8837b56 None None] MFA Rules not processed for user `0dc17670edbb4cc797103b0d41835bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:55.246420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-03a1fa0f-330e-46d5-ae54-9b1fc8837b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.333875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cb3ff-4f71-481f-8212-58d986a2ec6f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:55.343582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:55.347931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:55.347931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:55.347931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:55.369947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:55.373683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:55.407283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 818/1645] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:55.408628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:55.410460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.426404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 828/1646] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:19:55.438641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:55.438916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:55.439250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:55.472600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] MFA Rules not processed for user `0dc17670edbb4cc797103b0d41835bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:55.473945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.492042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 829/1647] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:55.584971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442513e-b99f-4c6a-8668-a8265fede504 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:55.605919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:55.606863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:55.607284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:55.607704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:55.610594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.711308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:55.712826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:55.805190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:55.806710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:55.842484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:55.843564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 819/1648] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:55.908589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69aee3-2f55-4546-bd3f-ba7612c63084 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:55.934333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:55.934333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:55.934333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:55.934333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:55.950610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:55.952177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc90073b-6904-4057-848b-65634d4e6882 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:55.978696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 830/1649] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:19:56.144162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:56.165033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795b623-0028-4a6c-abe6-9ad322281baf service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:56.186512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:56.187437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:56.187820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:56.189099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:56.192534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:56.341237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:56.342738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0f021-5b62-4ddc-b085-12a6fb876932 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:56.357835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 820/1650] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:57.486901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:57.488227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:57.488896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:57.491235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 831/1651] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 15:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:19:57.505082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:57.505695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:57.505695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:57.533057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:57.577181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:57.579119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:57.587430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 821/1652] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:19:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:57.690077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:57.692176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:57.720388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56e5ed-53a8-44a6-be23-639e8c50e159 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:57.734238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:57.734545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:57.734910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:57.735190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:57.735465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:57.736280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:57.751346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:57.752855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:57.786907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1b49-21d9-4efe-a008-06361cce3050 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:57.791628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:57.792054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 832/1653] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:57.793441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:57.793441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:57.793441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] PATH_INFO: `/v3/projects/1767066f43cc482bba115ccb2e1c1b1e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:57.796824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1767066f43cc482bba115ccb2e1c1b1e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:57.798065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: WARNING py.warnings [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:57.798065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: warnings.warn(msg) May 24 15:19:57.798065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: May 24 15:19:57.799540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:57.962627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:57.965171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:57.999265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1767066f43cc482bba115ccb2e1c1b1e', 'request_id': 'req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.009236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fa390-73c3-410c-99b2-6af9a155607d service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:58.017694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1767066f43cc482bba115ccb2e1c1b1e', 'request_id': 'req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.033711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1767066f43cc482bba115ccb2e1c1b1e', 'request_id': 'req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.073216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] The token cache is being invalidate because project 1767066f43cc482bba115ccb2e1c1b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:19:58.073767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.075653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.083567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 822/1654] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:19:57 2022] DELETE /identity/v3/projects/1767066f43cc482bba115ccb2e1c1b1e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:19:58.084865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:58.093395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:58.094150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:58.095487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:58.095983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:58.096561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] PATH_INFO: `/v3/users/8120c09231af4074a4d52db426e4e4cc` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:58.097971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8120c09231af4074a4d52db426e4e4cc)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:58.099206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: WARNING py.warnings [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 15:19:58.099206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: warnings.warn(msg) May 24 15:19:58.099206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: May 24 15:19:58.100897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:58.112275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fca89e2-c0d0-4ce3-bb4d-d989a564ead4 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:58.128138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:58.129252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:58.129650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:58.130025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] PATH_INFO: `/v3/users/f901441f65c84742b3007edd40a2b135` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:58.131548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f901441f65c84742b3007edd40a2b135)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:58.133694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:58.245410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f901441f65c84742b3007edd40a2b135'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.247132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f901441f65c84742b3007edd40a2b135'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.248060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f901441f65c84742b3007edd40a2b135'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.267030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f901441f65c84742b3007edd40a2b135'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.309164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-dda4dae0-4790-45b9-8209-7d5f71650642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f901441f65c84742b3007edd40a2b135'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.334038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 823/1655] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:19:58 2022] DELETE /identity/v3/users/f901441f65c84742b3007edd40a2b135 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:19:58.338398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8120c09231af4074a4d52db426e4e4cc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.342254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8120c09231af4074a4d52db426e4e4cc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.344522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8120c09231af4074a4d52db426e4e4cc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.369802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8120c09231af4074a4d52db426e4e4cc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.388070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8120c09231af4074a4d52db426e4e4cc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:19:58.460636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 833/1656] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:19:57 2022] DELETE /identity/v3/users/8120c09231af4074a4d52db426e4e4cc => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:19:58.466370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128471f-03d7-4b8e-9272-fa88bbd7a617 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:58.466597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128471f-03d7-4b8e-9272-fa88bbd7a617 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:58.466837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128471f-03d7-4b8e-9272-fa88bbd7a617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:58.513761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d128471f-03d7-4b8e-9272-fa88bbd7a617 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:58.516923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d128471f-03d7-4b8e-9272-fa88bbd7a617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:58.630474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:58.674185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 824/1657] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:19:58.731715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:58.733455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:58.844483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:19:58.891301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a23fb1-4b3a-42a8-b7ea-128f0cae25f2 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:58.908332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:58.909013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:58.909300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:58.909580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:58.934573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:58.939382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f244fe-7ac9-443a-8f87-6ee34e065b4f service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:58.971607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 834/1658] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:19:59.314848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef714a-638b-40d8-9514-49728deba373 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:59.315194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef714a-638b-40d8-9514-49728deba373 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:59.315473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef714a-638b-40d8-9514-49728deba373 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:59.317050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 825/1659] 10.222.0.45 () {54 vars in 960 bytes} [Tue May 24 15:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:19:59.322473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:59.322926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:59.323396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:59.349929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:19:59.352974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:59.559792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 835/1660] 10.222.0.45 () {58 vars in 1057 bytes} [Tue May 24 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:19:59.578690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef714a-638b-40d8-9514-49728deba373 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:19:59.621872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef714a-638b-40d8-9514-49728deba373 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:19:59.637981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:19:59.638923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:19:59.639334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:19:59.640053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:19:59.651254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:19:59.670599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:19:59.674564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0aa84a-841b-4777-806f-875143d41e61 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:19:59.700539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 826/1661] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:00.075359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:00.086347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:00.112221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26880283-9f2b-400b-ad37-2901639ee893 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:00.135689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:00.139257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:00.139534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:00.139812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] PATH_INFO: `/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:00.145227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62e99d527c5247bcb2ea4e9fdf7feb8b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:00.150552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:00.291257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62e99d527c5247bcb2ea4e9fdf7feb8b', 'request_id': 'req-6a90b796-7958-407b-9692-28800ca3bbc8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:00.303714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62e99d527c5247bcb2ea4e9fdf7feb8b', 'request_id': 'req-6a90b796-7958-407b-9692-28800ca3bbc8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:00.318327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62e99d527c5247bcb2ea4e9fdf7feb8b', 'request_id': 'req-6a90b796-7958-407b-9692-28800ca3bbc8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:00.379203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] The token cache is being invalidate because project 62e99d527c5247bcb2ea4e9fdf7feb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:00.379573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:00.382124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6a90b796-7958-407b-9692-28800ca3bbc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:00.385864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 836/1662] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:00 2022] DELETE /identity/v3/projects/62e99d527c5247bcb2ea4e9fdf7feb8b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:00.391942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:00.391942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:00.391942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:00.476583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:00.478344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:00.688338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 827/1663] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:00.707299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:00.707978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:00.708596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:00.754681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:00.757828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:00.780112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 837/1664] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:00.785549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:00.799597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1151100-05b9-4b6f-9e62-ea336abf1fc4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:00.819987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:00.821053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:00.821491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:00.821901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:00.822962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:00.824136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:00.826524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:00.842002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 828/1665] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:00.846761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:00.857755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aea5d-efba-49c4-bb33-5021ed550607 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:00.876083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:00.876871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:00.877168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:00.877469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:00.880015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:00.889506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:00.948374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:00.954608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 838/1666] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:00 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:00.965013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4c219-d486-4afa-9199-1749aa1503e2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:01.119213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:01.121465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.122056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.122702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.123920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:01.127197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:01.181782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 829/1667] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:00 2022] POST /identity/v3/users => generated 282 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:01.185509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:01.207873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e348d-79c1-4a5a-a18e-67eacaad9aaf admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:01.217475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:01.219330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.219919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.224681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.225803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:01.227861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:01.261755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 839/1668] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:01.262660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:01.285855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aa89e-a817-45ec-9479-0b4976f337a8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:01.305739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:01.307607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.308330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.308626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.311835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:01.318914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:01.357552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 830/1669] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:01.361069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:01.384584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f5691-87ee-42b7-b5c2-fd998c31b89b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:01.397055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:01.397898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.398202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.398477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] PATH_INFO: `/v3/projects/0e6325f3997844898c219c519e65175f/users/2cfe7773c4a34aa194e3131c4378b48f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.458180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:01.460646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:01.516002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18760270-428d-4c7f-86fe-a3bea18086eb admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:01.545483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:01.547179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.550048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.550891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] PATH_INFO: `/v3/projects/cb5a1867355149cd844bf31c563b9866` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.555861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5a1867355149cd844bf31c563b9866)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:01.564600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:01.717047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:01.769235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5a1867355149cd844bf31c563b9866', 'request_id': 'req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:01.781855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5a1867355149cd844bf31c563b9866', 'request_id': 'req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:01.801727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5a1867355149cd844bf31c563b9866', 'request_id': 'req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:01.831574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6325f3997844898c219c519e65175f, user_id=2cfe7773c4a34aa194e3131c4378b48f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:01.841258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036869be-b359-4e6e-929f-1f64d7242428 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:01.864953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:01.878050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] The token cache is being invalidate because project cb5a1867355149cd844bf31c563b9866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:01.879370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:01.884484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 840/1670] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:01 2022] PUT /identity/v3/projects/0e6325f3997844898c219c519e65175f/users/2cfe7773c4a34aa194e3131c4378b48f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:01.884798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4210b784-b9fb-47f3-84fe-e1ddd7d5f508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:01.891942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 831/1671] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:01 2022] DELETE /identity/v3/projects/cb5a1867355149cd844bf31c563b9866 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:01.894436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.895910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.896476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.899606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19209e65-e0d6-4939-93dc-4b61910dd856 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:01.900051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19209e65-e0d6-4939-93dc-4b61910dd856 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:01.900393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19209e65-e0d6-4939-93dc-4b61910dd856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:01.941477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] MFA Rules not processed for user `2cfe7773c4a34aa194e3131c4378b48f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:01.943495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:01.956801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-19209e65-e0d6-4939-93dc-4b61910dd856 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:01.957970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-19209e65-e0d6-4939-93dc-4b61910dd856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:02.037037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 841/1672] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:02.044687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.046812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:02.165898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd9fbf-7d9c-437f-959e-e63390a37546 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.184402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.185640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.186251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.186251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.191799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.200502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.224713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 832/1673] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:02.232687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.233379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.233796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.257041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:02.258474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:02.270274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 833/1674] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:02.275581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.286968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec03797-a1d8-455e-8004-c6016d76fa7f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.293983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.295306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.295421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.295930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.296724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:02.297031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.298848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.306062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 842/1675] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:02 2022] POST /identity/v3/projects => generated 365 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:02.307877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.327104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 834/1676] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:02.328070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.353249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8a48a-59be-4208-9f92-fc07acaa9f8a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.478866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528f333-bd1d-4333-a8ab-2fd4d01fb0ec admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.493951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.494937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.496151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.497721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.499083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.507730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.516644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.517328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.517613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.517887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.518586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.520543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.564755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 843/1677] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:02 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:02.573177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.587254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:02.593965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 835/1678] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:02 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:02.594885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.614410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89064-cf60-4275-8c66-f23415517728 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.705495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.706359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.706649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.706929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.707813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.710290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.718654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:02.736767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 844/1679] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:02.739947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.760981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e7d7f-ef05-462e-b872-cd4b150255c7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.776391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.779663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.780221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.780646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] PATH_INFO: `/v3/projects/97a316f6d6f54d64b77deed3ac580653/users/f6abee07217443e49411fd04994a640a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.830687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a316f6d6f54d64b77deed3ac580653, user_id=f6abee07217443e49411fd04994a640a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.835207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.840633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0bd39-e818-4413-8922-c001bc77f99d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:02.859860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:02.860821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:02.861270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:02.861552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:02.863461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:02.867525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 845/1680] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:02 2022] PUT /identity/v3/projects/97a316f6d6f54d64b77deed3ac580653/users/f6abee07217443e49411fd04994a640a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:02.874654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:02.883363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:02.959859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 836/1681] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:02 2022] POST /identity/v3/users => generated 286 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:02.963839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:03.020145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bd64c-6973-4bf4-970e-565c9089785d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.120772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bc7f2-f204-467f-885f-38c3fd724eac admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.187858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.195382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.195792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.201959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.204520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:03.207797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:03.243830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.244495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.244859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.245035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.245802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:03.247543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:03.249463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 846/1682] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:03.253871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:03.284393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 837/1683] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:03.285420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:03.296977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f776a7-8d62-44a7-899b-b675fe0fb348 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.328224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.330339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfe8c8-944a-4a04-9367-ef1bed0c8756 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.330508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.333352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.333879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] PATH_INFO: `/v3/projects/97a316f6d6f54d64b77deed3ac580653/users/f6abee07217443e49411fd04994a640a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.343233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a316f6d6f54d64b77deed3ac580653, user_id=f6abee07217443e49411fd04994a640a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:03.346247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84b3f0-3011-4c0d-82f9-37e60815fbd9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:03.374269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.375011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.378523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.378836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.381308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:03.383009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:03.387073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 847/1684] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:03 2022] PUT /identity/v3/projects/97a316f6d6f54d64b77deed3ac580653/users/f6abee07217443e49411fd04994a640a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:03.402900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.403661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.405715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.426611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 838/1685] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:03.431726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:03.467435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0822994a-055b-4b6c-bd0f-ce7549006fa2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.481691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] MFA Rules not processed for user `f6abee07217443e49411fd04994a640a`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:03.483424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:03.617219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:03.622972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.624203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.624554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.624946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] PATH_INFO: `/v3/projects/3cfcb23e870f4778b2f947f2e100eb91/users/127b2d5c6d5f4dda9e89a00db892e0d4/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.661757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfcb23e870f4778b2f947f2e100eb91, user_id=127b2d5c6d5f4dda9e89a00db892e0d4, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:03.667375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d885c37-62d4-43fc-ac24-d27a8277cf42 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:03.686043 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 848/1686] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:03.711917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 839/1687] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:03 2022] PUT /identity/v3/projects/3cfcb23e870f4778b2f947f2e100eb91/users/127b2d5c6d5f4dda9e89a00db892e0d4/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:03.713518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:03.715465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89674-1449-4fae-88cb-b82a0b404f52 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.715761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89674-1449-4fae-88cb-b82a0b404f52 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.716043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d89674-1449-4fae-88cb-b82a0b404f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:03.716496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:03.813747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-48d89674-1449-4fae-88cb-b82a0b404f52 None None] MFA Rules not processed for user `127b2d5c6d5f4dda9e89a00db892e0d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:03.815698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-48d89674-1449-4fae-88cb-b82a0b404f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:03.851587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:03.873695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b52eec-09d4-420e-b81d-2319994133e4 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:03.992989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:03.993581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:03.993859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:03.994127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.003147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 840/1688] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:04.004615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00d42-7af8-404e-bffe-a6389de9e133 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.004906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00d42-7af8-404e-bffe-a6389de9e133 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.006586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a00d42-7af8-404e-bffe-a6389de9e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.063282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-78a00d42-7af8-404e-bffe-a6389de9e133 None None] MFA Rules not processed for user `2d8a7414fd254ec981d35b417dc373cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:04.064542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-78a00d42-7af8-404e-bffe-a6389de9e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:04.120386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.122383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.134320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 849/1689] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:04.135345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.165953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 841/1690] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:04.177652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.177942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.178222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.197382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] MFA Rules not processed for user `2d8a7414fd254ec981d35b417dc373cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:04.198690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:04.212358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 842/1691] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:04.213095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:04.219101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.223117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:04.232624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb5a19-fbba-4047-a58e-31d99954b7d3 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.240986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.242572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.242891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.243234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.243897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.248291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.292341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 850/1692] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:04 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:04.294577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.348405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa034-0f5c-424d-8dda-b726d77d52b9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.420564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:04.420866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3d1f2-e7ba-47da-be8a-912c96732ad1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.426450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.427352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.428205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.428610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.429555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.431601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.481677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 851/1693] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:04 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:04.483266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.514808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbc743-4aa9-475b-a97f-48834a824424 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.521654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.522687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.523167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.523888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.529799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.531282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.532795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.534137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.535061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.542406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.584122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 852/1694] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:04.589873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.610723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea879d-c632-40f7-809c-258876904ae5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.623100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.624065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.624420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.624784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] PATH_INFO: `/v3/projects/582b5bc60de140e09d70d7421b698bb8/users/e9704696bd7444ad9c4a88c99481f65c/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.643069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.644993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.658106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 843/1695] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:04.661246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582b5bc60de140e09d70d7421b698bb8, user_id=e9704696bd7444ad9c4a88c99481f65c, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.664247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.692561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 853/1696] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:04 2022] PUT /identity/v3/projects/582b5bc60de140e09d70d7421b698bb8/users/e9704696bd7444ad9c4a88c99481f65c/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:04.696351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.731711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0a527-5b20-46e7-ad61-0181d5658674 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.844545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.845854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.846203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.846488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.847230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.848963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197a876-a72b-4825-b80e-1dcae52d3f31 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.864652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 844/1697] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:04 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:04.868546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:04.884902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe4ec7-e9ad-4b6d-ab57-542d4723ce05 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:04.898102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:04.899051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.899333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.899609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] PATH_INFO: `/v3/projects/582b5bc60de140e09d70d7421b698bb8/users/e9704696bd7444ad9c4a88c99481f65c/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:04.907575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582b5bc60de140e09d70d7421b698bb8, user_id=e9704696bd7444ad9c4a88c99481f65c, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:04.922772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:04.973234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 854/1698] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:04 2022] PUT /identity/v3/projects/582b5bc60de140e09d70d7421b698bb8/users/e9704696bd7444ad9c4a88c99481f65c/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:04.984016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:04.984617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:04.985107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:05.041819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:05.092884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] MFA Rules not processed for user `e9704696bd7444ad9c4a88c99481f65c`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:05.094208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:05.206898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c3fdd4-c815-4f69-96f7-cada8f8157c5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:05.213247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:05.215409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:05.215833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:05.216241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] PATH_INFO: `/v3/users/2cfe7773c4a34aa194e3131c4378b48f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:05.221209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cfe7773c4a34aa194e3131c4378b48f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:05.223228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:05.280636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:05.285078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:05.314582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 845/1699] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:05.342962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:05.349690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:05.366871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cfe7773c4a34aa194e3131c4378b48f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:05.369520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cfe7773c4a34aa194e3131c4378b48f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:05.373173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cfe7773c4a34aa194e3131c4378b48f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:05.399362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cfe7773c4a34aa194e3131c4378b48f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:05.423544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-86747a9a-25d8-4974-ae40-0b69a3f8a04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cfe7773c4a34aa194e3131c4378b48f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:05.464031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 855/1700] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:05 2022] DELETE /identity/v3/users/2cfe7773c4a34aa194e3131c4378b48f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:05.466456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:05.466955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:05.467435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:05.514088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baafed-9827-455f-8994-807bab73b9d2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:05.526363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:05.527709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:05.666173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:05.668628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:05.669421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:05.669837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:05.672216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:05.674127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 856/1701] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:05.698164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:05.700388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:05.800449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:05.802308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb57f8f-5795-487e-b340-bba0e581acaa service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:05.820710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 846/1702] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:05.835974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584cd825-b165-42b6-8533-9b2ab2b52671 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:05.853096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:05.854509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:05.854509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:05.854716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:05.885339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:05.887359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f21212-bd8c-4a41-bf0c-128fab2e9e76 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:05.918400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 857/1703] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:06.517779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:06.518444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:06.518603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:06.570250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:06.575271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:06.578290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:06.579642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:06.579932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:06.672521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] MFA Rules not processed for user `2d8a7414fd254ec981d35b417dc373cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:06.677782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:06.831072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:06.834032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 847/1704] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:06.836026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:06.841982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:06.898968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:06.905538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1084d-088d-4074-932b-453a860f88a3 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:06.911833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:06.914475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:06.914823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:06.915109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] PATH_INFO: `/v3/users/127b2d5c6d5f4dda9e89a00db892e0d4` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:06.916500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127b2d5c6d5f4dda9e89a00db892e0d4)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:06.919007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:06.946877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 858/1705] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:06.947669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:06.951492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:06.982491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfeb9c-4cb3-47b9-bb45-2a9aea54eb52 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:07.014090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:07.017950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:07.018282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:07.018554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] PATH_INFO: `/v3/users/0905caa551494b479b47da44cc0b7de7` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:07.022244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0905caa551494b479b47da44cc0b7de7)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:07.034497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127b2d5c6d5f4dda9e89a00db892e0d4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.035402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127b2d5c6d5f4dda9e89a00db892e0d4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.036018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127b2d5c6d5f4dda9e89a00db892e0d4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.041288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:07.051916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127b2d5c6d5f4dda9e89a00db892e0d4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.096679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127b2d5c6d5f4dda9e89a00db892e0d4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.150422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 848/1706] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:06 2022] DELETE /identity/v3/users/127b2d5c6d5f4dda9e89a00db892e0d4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:07.150422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:07.153769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.180698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0905caa551494b479b47da44cc0b7de7'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.182367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0905caa551494b479b47da44cc0b7de7'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.192507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0905caa551494b479b47da44cc0b7de7'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.215933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0905caa551494b479b47da44cc0b7de7'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.233763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-546f4447-55f9-4547-a9a5-09ec68b3f3d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0905caa551494b479b47da44cc0b7de7'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:07.291599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 859/1707] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:06 2022] DELETE /identity/v3/users/0905caa551494b479b47da44cc0b7de7 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:07.297648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944f00-0425-4698-ad98-3b5f3e7081b7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:07.301297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944f00-0425-4698-ad98-3b5f3e7081b7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:07.301749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3944f00-0425-4698-ad98-3b5f3e7081b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:07.338065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe537-d8ff-485d-8609-2dd2dd26b676 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:07.359080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a3944f00-0425-4698-ad98-3b5f3e7081b7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:07.366330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a3944f00-0425-4698-ad98-3b5f3e7081b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.538583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:07.539831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:07.540763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:07.540763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:07.545095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.609874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:07.654441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 860/1708] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:07.654441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45623ab-5012-49ad-9a32-ba701e9f968d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:07.654441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45623ab-5012-49ad-9a32-ba701e9f968d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:07.654441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45623ab-5012-49ad-9a32-ba701e9f968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:07.677903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a45623ab-5012-49ad-9a32-ba701e9f968d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:07.679243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a45623ab-5012-49ad-9a32-ba701e9f968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.692344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 861/1709] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:07.694785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:07.695327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:07.695712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:07.737173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] MFA Rules not processed for user `2befe74ad06542b785d326101fc06d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:07.738791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.767859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:07.771941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:07.818350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 849/1710] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:20:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:20:07.820077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:07.825016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:07.930985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:08.114165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb2ece-9952-4bcc-9965-a083f4e5a91f service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:08.119961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 862/1711] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:08.121153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:08.132411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:08.133095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:08.134545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:08.134743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:08.135032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:08.166973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:08.179243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:08.200558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b961b05-f32d-4873-9a11-f16252aeb0dd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:08.222433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 850/1712] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:08.223822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:08.237255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:08.238895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:08.241383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:08.241737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] PATH_INFO: `/v3/projects/0e6325f3997844898c219c519e65175f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:08.251862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6325f3997844898c219c519e65175f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:08.259517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:08.269368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff7ac7-19fe-4224-a50b-e8b6968e768c service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:08.299305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:08.299979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:08.300414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:08.300606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:08.361216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:08.362340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:08.432483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 851/1713] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:08.440606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:08.447032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:08.578129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6325f3997844898c219c519e65175f', 'request_id': 'req-ba383bf6-da68-4fbf-9867-1ac51e6f336d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:08.620971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6325f3997844898c219c519e65175f', 'request_id': 'req-ba383bf6-da68-4fbf-9867-1ac51e6f336d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:08.643650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6325f3997844898c219c519e65175f', 'request_id': 'req-ba383bf6-da68-4fbf-9867-1ac51e6f336d'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:08.739019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] The token cache is being invalidate because project 0e6325f3997844898c219c519e65175f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:08.746100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:08.747853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:08.767074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 863/1714] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:08 2022] DELETE /identity/v3/projects/0e6325f3997844898c219c519e65175f => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:08.772358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:08.776490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:08.955068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc08e2c-e8a3-461c-9eda-ff94cda9fd35 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:09.181284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:09.238366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba383bf6-da68-4fbf-9867-1ac51e6f336d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:09.262288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:09.263218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:09.263493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:09.263769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] PATH_INFO: `/v3/users/f6abee07217443e49411fd04994a640a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:09.264344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:09.265175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:09.265600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:09.265829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6abee07217443e49411fd04994a640a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:09.266255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:09.267794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:09.276334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:09.478801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6abee07217443e49411fd04994a640a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:09.483410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6abee07217443e49411fd04994a640a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:09.486341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6abee07217443e49411fd04994a640a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:09.520611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6abee07217443e49411fd04994a640a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:09.567794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6abee07217443e49411fd04994a640a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:09.568947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:09.570567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:09.577933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:09.625167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 852/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 1192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:09.686001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 864/1716] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:08 2022] DELETE /identity/v3/users/f6abee07217443e49411fd04994a640a => generated 0 bytes in 915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:11.622791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:11.625476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:11.647727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112ee0b-4865-4879-8b6e-4beda257d64f service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:11.746118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:11.749360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:11.843559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:11.848485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:11.848863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:11.850561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] PATH_INFO: `/v3/projects/3cfcb23e870f4778b2f947f2e100eb91` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:11.855869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfcb23e870f4778b2f947f2e100eb91)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:11.860863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:11.947509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904da04-72d8-4051-a542-c93ad1ebe3fa admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:11.966040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:11.967278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:11.969362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:11.969761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] PATH_INFO: `/v3/projects/97a316f6d6f54d64b77deed3ac580653` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:11.971934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a316f6d6f54d64b77deed3ac580653)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:11.975443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:12.099126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfcb23e870f4778b2f947f2e100eb91', 'request_id': 'req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.125429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfcb23e870f4778b2f947f2e100eb91', 'request_id': 'req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.152239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfcb23e870f4778b2f947f2e100eb91', 'request_id': 'req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.172482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] The token cache is being invalidate because project 3cfcb23e870f4778b2f947f2e100eb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:12.173237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.174073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.176207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 853/1717] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:11 2022] DELETE /identity/v3/projects/3cfcb23e870f4778b2f947f2e100eb91 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:12.180178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:12.184605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:12.226003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:12.239806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a316f6d6f54d64b77deed3ac580653', 'request_id': 'req-31400ef6-f47b-4405-9d19-a2634a0624b0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.253382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a316f6d6f54d64b77deed3ac580653', 'request_id': 'req-31400ef6-f47b-4405-9d19-a2634a0624b0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.275536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a316f6d6f54d64b77deed3ac580653', 'request_id': 'req-31400ef6-f47b-4405-9d19-a2634a0624b0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.300486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] The token cache is being invalidate because project 97a316f6d6f54d64b77deed3ac580653 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:12.308254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.309805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.311993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 865/1718] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:11 2022] DELETE /identity/v3/projects/97a316f6d6f54d64b77deed3ac580653 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:12.312841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:12.314303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:12.426439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31400ef6-f47b-4405-9d19-a2634a0624b0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:12.431822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:12.432997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:12.433295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:12.434127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] PATH_INFO: `/v3/users/83b2f2db9bd24ab9954693480d81df44` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:12.435307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b2f2db9bd24ab9954693480d81df44)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:12.437626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:12.460517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:12.464815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5479e9-58f4-4d7c-aef3-ba00ae99437a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:12.515015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:12.515015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:12.515015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:12.515015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] PATH_INFO: `/v3/users/e9704696bd7444ad9c4a88c99481f65c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:12.517746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9704696bd7444ad9c4a88c99481f65c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:12.519888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:12.578317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b2f2db9bd24ab9954693480d81df44'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.581394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b2f2db9bd24ab9954693480d81df44'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.583990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b2f2db9bd24ab9954693480d81df44'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.608128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b2f2db9bd24ab9954693480d81df44'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.626895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-8adf17bc-c437-4212-bcb9-a59b00e48f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b2f2db9bd24ab9954693480d81df44'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.717178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9704696bd7444ad9c4a88c99481f65c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.725614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9704696bd7444ad9c4a88c99481f65c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.726190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 866/1719] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:12 2022] DELETE /identity/v3/users/83b2f2db9bd24ab9954693480d81df44 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:12.727622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9704696bd7444ad9c4a88c99481f65c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.741229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4682-2153-437f-a06a-f2080ebd3a99 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:12.741677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4682-2153-437f-a06a-f2080ebd3a99 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:12.741986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f4682-2153-437f-a06a-f2080ebd3a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:12.760503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9704696bd7444ad9c4a88c99481f65c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.791831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6fde816b-f34e-436c-966b-4d774335bee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9704696bd7444ad9c4a88c99481f65c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:12.819859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 854/1720] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:12 2022] DELETE /identity/v3/users/e9704696bd7444ad9c4a88c99481f65c => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:12.821368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:12.824109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:12.824523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:12.865258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-062f4682-2153-437f-a06a-f2080ebd3a99 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:12.866689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-062f4682-2153-437f-a06a-f2080ebd3a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:12.873799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:12.877897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.075399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 855/1721] 10.222.0.48 () {58 vars in 1015 bytes} [Tue May 24 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:13.076545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.080045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.139997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 867/1722] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:13.139997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1238a-213e-4651-8cc8-4d95e30e21b2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.140469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1238a-213e-4651-8cc8-4d95e30e21b2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.140469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1238a-213e-4651-8cc8-4d95e30e21b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.168789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-71d1238a-213e-4651-8cc8-4d95e30e21b2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:13.170070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-71d1238a-213e-4651-8cc8-4d95e30e21b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.200641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:13.202635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 868/1723] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:13.203942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.204347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.204689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.272195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853afe9d-fb82-40c2-a787-b8ac16eda8b9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.286192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.288508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.290961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.291351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] PATH_INFO: `/v3/projects/e3d7790a5da14f9090ecbfc53c3f7b49` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.293193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d7790a5da14f9090ecbfc53c3f7b49)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:13.296141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:13.298252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:13.304686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.347345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 869/1724] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:13.347345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.347345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.386523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78697b3e-6e88-4f06-a4f1-ff35ce878a29 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.407881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.408766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.409055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.409330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.436601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d7790a5da14f9090ecbfc53c3f7b49', 'request_id': 'req-a5d8784d-31a4-4b78-b7b2-031d1c777116'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:13.444999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:13.446657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:13.453958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d7790a5da14f9090ecbfc53c3f7b49', 'request_id': 'req-a5d8784d-31a4-4b78-b7b2-031d1c777116'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:13.479435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d7790a5da14f9090ecbfc53c3f7b49', 'request_id': 'req-a5d8784d-31a4-4b78-b7b2-031d1c777116'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:13.490247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 870/1725] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:13.491383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.503129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] The token cache is being invalidate because project e3d7790a5da14f9090ecbfc53c3f7b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:13.503347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:13.504202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-a5d8784d-31a4-4b78-b7b2-031d1c777116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:13.511793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 856/1726] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:13 2022] DELETE /identity/v3/projects/e3d7790a5da14f9090ecbfc53c3f7b49 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:13.513810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676c69-73bf-4477-b638-d384ac624f14 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.514335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676c69-73bf-4477-b638-d384ac624f14 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.514624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36676c69-73bf-4477-b638-d384ac624f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.522040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804e6205-18d4-4daa-bb49-338592329b64 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.550129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-36676c69-73bf-4477-b638-d384ac624f14 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:13.551617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-36676c69-73bf-4477-b638-d384ac624f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.672205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 857/1727] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:13.674458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.674755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.675432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.698226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:13.700045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.714783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.718166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.718166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.718166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.718166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:13.718932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 858/1728] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:13.720389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.724682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.737866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:13.739546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:13.762169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:13.778669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 871/1729] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:13.779590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.824854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb194a-95a2-40a7-974c-569e0a60049d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.842864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.843557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.843930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.844214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.844984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:13.845293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:13.846918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:13.895213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 872/1730] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:13.895213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:13.895213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:13.937776 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0662af-34ec-494b-93d3-53e24013cec6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.949508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68813c1-c933-4ab9-bad4-6c46de26ec8d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:13.950988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.951680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.952073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.952410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.955255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:13.961077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:13.978171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:13.981404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:13.981698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:13.981982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:13.990638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:14.027348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 873/1731] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:13 2022] POST /identity/v3/projects => generated 355 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:14.029548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.039099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.044892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.086378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 859/1732] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:14.089812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.134506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda38b0e-3cc6-467e-b0da-cd0396d422a1 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.165655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:14.235457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf425-3c65-41f0-9a4d-f6c9266095ed admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.249975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.253460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.254716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.256649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.258687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.263787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.314498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.315440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.315803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.316373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.318622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.330460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.362147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 874/1733] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:14 2022] POST /identity/v3/projects => generated 375 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:14.363387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.385008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 860/1734] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:14 2022] POST /identity/v3/users => generated 278 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:14.389933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.418112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a20e03-33ea-47a7-a85a-62a0576a15a2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.525608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.526527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.526904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.527656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.528672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.530522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.558284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 861/1735] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:14.560070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.598511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb00f7e-7948-4cc2-9180-70898ad55564 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.606238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.607235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.607698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.608410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.609361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.616851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14770f-8039-485f-8ea0-b9ba51ae7afb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.676264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0066-222e-4c91-9f0e-619b5932c6cb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.684141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 862/1736] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:14.686989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3cf0-dda5-4b78-9aae-038f531dbd2b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.690933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3cf0-dda5-4b78-9aae-038f531dbd2b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.693101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e3cf0-dda5-4b78-9aae-038f531dbd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.706376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.707122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.707347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.707626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.708365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.711808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.777714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 875/1737] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:14 2022] POST /identity/v3/users => generated 288 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:14.778579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.810477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f79e3cf0-dda5-4b78-9aae-038f531dbd2b None None] MFA Rules not processed for user `40db5795ce7e4c5bbf06fb81ce404e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:14.813193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f79e3cf0-dda5-4b78-9aae-038f531dbd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:14.829434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c489f-e761-44ae-b990-a98160ec8b99 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:14.841339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:14.842357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:14.842645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:14.842973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] PATH_INFO: `/v3/projects/8e90ebb6ca4441a0a0c6df5dcff8eb4e/users/9d58372887b14787a831b7d3aa26119c/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:14.918971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e90ebb6ca4441a0a0c6df5dcff8eb4e, user_id=9d58372887b14787a831b7d3aa26119c, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:14.928860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:14.976837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 876/1738] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:14 2022] PUT /identity/v3/projects/8e90ebb6ca4441a0a0c6df5dcff8eb4e/users/9d58372887b14787a831b7d3aa26119c/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:14.984850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:14.993341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 863/1739] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:15.008097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:15.008572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:15.008858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:15.036774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:15.160340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] MFA Rules not processed for user `9d58372887b14787a831b7d3aa26119c`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:15.167870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:15.176071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:15.239030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4cb4e6-7fc5-4064-a312-90685c873648 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:15.253195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:15.254065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:15.254455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:15.254929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:15.255978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:15.258636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:15.297598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 877/1740] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:15.302927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:15.312453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:15.672188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28504261-ce04-43ef-b483-d1a8855a50d7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:15.689958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:15.700732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:15.701256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:15.701859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:15.862897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 864/1741] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:14 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 869 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:15.867272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:15.898005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319120ac-24c7-4202-87ba-21e35f5d2317 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:15.903373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:15.904069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:15.904385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:15.904662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:15.905569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:15.907269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:15.927478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 865/1742] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:15.928479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:15.942681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc180693-ce96-4758-ab0d-747c25a6bd6a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:15.953883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:15.954485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:15.954961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:15.954961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:15.956817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:15.960161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:15.961114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:15.967565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:15.994545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 878/1743] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:15.996335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:16.046767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 866/1744] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:15 2022] POST /identity/v3/projects => generated 355 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:16.053129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:16.072091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02d1c5-d2ce-4390-9695-8930c36bd125 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:16.238070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea61463-5453-410b-b0c2-15df9a889a1d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:16.272590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:16.276621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:16.276979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:16.284326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:16.285250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:16.287342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:16.366632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 867/1745] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:16 2022] POST /identity/v3/users => generated 284 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:16.367255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:16.370606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:16.412661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62d986-ec54-4dbe-b475-54e5c4670824 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:16.423131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:16.425986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:16.426771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:16.428782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] PATH_INFO: `/v3/projects/c2b0f66ad6bb4ece98ea1859904d2a11/users/9fe7d2ed13994dcf91884ac89dda3a0a/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:16.433558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:16.437132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:16.437790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:16.438080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:16.441441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:16.449699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:16.511475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2b0f66ad6bb4ece98ea1859904d2a11, user_id=9fe7d2ed13994dcf91884ac89dda3a0a, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:16.520538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 868/1746] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:16.521521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:16.523407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45702e-6c21-4836-b6ed-a7c19726ffb7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:16.557693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393c5a4-9f06-4854-9273-ac495a3b0f7c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:16.582675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 879/1747] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:15 2022] PUT /identity/v3/projects/c2b0f66ad6bb4ece98ea1859904d2a11/users/9fe7d2ed13994dcf91884ac89dda3a0a/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:16.595979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:16.597076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:16.597376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:16.707772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] MFA Rules not processed for user `9fe7d2ed13994dcf91884ac89dda3a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:16.709551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:16.782972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:16.817500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:16.819838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:16.822741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:16.823933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] PATH_INFO: `/v3/projects/d6babfc1283445d98e8b04f6521bce3e/users/da17193e2feb4f2ca5f144ad5d706b36/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:16.843230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6babfc1283445d98e8b04f6521bce3e, user_id=da17193e2feb4f2ca5f144ad5d706b36, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:16.847895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:16.905249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 869/1748] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:16 2022] PUT /identity/v3/projects/d6babfc1283445d98e8b04f6521bce3e/users/da17193e2feb4f2ca5f144ad5d706b36/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:16.907485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:16.911147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 880/1749] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:17.042787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c85f0-7830-4762-950d-7f2e4a9b06f6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:17.050109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:17.051108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:17.051421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:17.051792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:17.052943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:17.055564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:17.074457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 870/1750] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:17.080213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:17.093768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5e98-3cf5-4dd8-96a8-bd7b362c4ea7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:17.103226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:17.111536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:17.112126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:17.112761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] PATH_INFO: `/v3/projects/d6babfc1283445d98e8b04f6521bce3e/users/da17193e2feb4f2ca5f144ad5d706b36/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:17.121702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6babfc1283445d98e8b04f6521bce3e, user_id=da17193e2feb4f2ca5f144ad5d706b36, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:17.138350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf305b-4577-4f53-bdee-d709649a94ae admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:17.197149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:17.200112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:17.220103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 881/1751] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:17 2022] PUT /identity/v3/projects/d6babfc1283445d98e8b04f6521bce3e/users/da17193e2feb4f2ca5f144ad5d706b36/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:17.232786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:17.233234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:17.233518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:17.341203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] MFA Rules not processed for user `da17193e2feb4f2ca5f144ad5d706b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:17.342464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:17.488874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4ed7a-c409-43f8-bf1e-ec0906131b5d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:17.754171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:17.755196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:17.755587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:17.756503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:17.781664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:17.798949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 882/1752] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:17 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 567 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:17.818319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:17.824682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:17.941593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:17.945411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:17.974540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 871/1753] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:17.978920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:17.982402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:17.983494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:18.132561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21b18d-ea83-40d5-a9ab-591369471f35 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:18.139493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:18.140603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:18.141577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:18.141950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] PATH_INFO: `/v3/projects/97174fe5a2f44eb48a6bbb17756601c6` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:18.143899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97174fe5a2f44eb48a6bbb17756601c6)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:18.146772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:18.173263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b394ee-29d7-4cea-b83c-9806e0b97deb service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:18.211477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:18.212396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:18.212679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:18.213095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] PATH_INFO: `/v3/projects/582b5bc60de140e09d70d7421b698bb8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:18.216742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=582b5bc60de140e09d70d7421b698bb8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:18.233703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:18.447139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97174fe5a2f44eb48a6bbb17756601c6', 'request_id': 'req-7b47112d-c57b-4cb4-a651-358651e5fe73'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.481029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97174fe5a2f44eb48a6bbb17756601c6', 'request_id': 'req-7b47112d-c57b-4cb4-a651-358651e5fe73'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.508710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97174fe5a2f44eb48a6bbb17756601c6', 'request_id': 'req-7b47112d-c57b-4cb4-a651-358651e5fe73'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.550515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] The token cache is being invalidate because project 97174fe5a2f44eb48a6bbb17756601c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:18.550884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.552837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582b5bc60de140e09d70d7421b698bb8', 'request_id': 'req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.567271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.568337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582b5bc60de140e09d70d7421b698bb8', 'request_id': 'req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.578080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 883/1754] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:17 2022] DELETE /identity/v3/projects/97174fe5a2f44eb48a6bbb17756601c6 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:18.583891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:18.592353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:18.594430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582b5bc60de140e09d70d7421b698bb8', 'request_id': 'req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.616894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] The token cache is being invalidate because project 582b5bc60de140e09d70d7421b698bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:18.617187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.617859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.619705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 872/1755] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:17 2022] DELETE /identity/v3/projects/582b5bc60de140e09d70d7421b698bb8 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:18.621410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:18.625016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:18.696967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:18.795073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c734f9-b8bd-4e95-b6e6-23f3a4db56d2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:18.825287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:18.826673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:18.831992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:18.832561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] PATH_INFO: `/v3/users/9fe7d2ed13994dcf91884ac89dda3a0a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:18.837028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fe7d2ed13994dcf91884ac89dda3a0a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:18.848490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:18.893501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47112d-c57b-4cb4-a651-358651e5fe73 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:18.982264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fe7d2ed13994dcf91884ac89dda3a0a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.988802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fe7d2ed13994dcf91884ac89dda3a0a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:18.989594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fe7d2ed13994dcf91884ac89dda3a0a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.005553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:19.006654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.007115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.007651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.011774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.021058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fe7d2ed13994dcf91884ac89dda3a0a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.093105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:19.100649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fe7d2ed13994dcf91884ac89dda3a0a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.128576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 873/1756] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:18 2022] DELETE /identity/v3/users/9fe7d2ed13994dcf91884ac89dda3a0a => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:19.129330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:19.130691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.229701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:19.234160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf2950-1b1b-43a2-8bc8-9232544fb197 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:19.271438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 884/1757] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:18 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:19.273649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787945-b25d-4537-aff2-f9c50dff81f3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.274637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787945-b25d-4537-aff2-f9c50dff81f3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.277163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787945-b25d-4537-aff2-f9c50dff81f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.313440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595ddde-a4eb-4466-9b2f-aa564adc76b6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:19.322585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:19.323637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.323924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.324200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] PATH_INFO: `/v3/users/7300fbec15694ab89839101334ed1522` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.329248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=7300fbec15694ab89839101334ed1522)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:19.331466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:19.341225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-40787945-b25d-4537-aff2-f9c50dff81f3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:19.344259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-40787945-b25d-4537-aff2-f9c50dff81f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.365878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 885/1758] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:19.367783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10121a0-e7c0-44ef-976b-68a8b3549fb4 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.368248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10121a0-e7c0-44ef-976b-68a8b3549fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.368557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10121a0-e7c0-44ef-976b-68a8b3549fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.433826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d10121a0-e7c0-44ef-976b-68a8b3549fb4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:19.436630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d10121a0-e7c0-44ef-976b-68a8b3549fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.482439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 886/1759] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:19.488686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7fd5-8abb-4bda-bf87-549be23e1d22 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.489150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7fd5-8abb-4bda-bf87-549be23e1d22 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.489436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7fd5-8abb-4bda-bf87-549be23e1d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.494049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7300fbec15694ab89839101334ed1522'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.498867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7300fbec15694ab89839101334ed1522'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.499671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7300fbec15694ab89839101334ed1522'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.522536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7300fbec15694ab89839101334ed1522'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.524740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:19.529542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f7ed7fd5-8abb-4bda-bf87-549be23e1d22 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:19.530974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f7ed7fd5-8abb-4bda-bf87-549be23e1d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.555624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 887/1760] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:19.557158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7300fbec15694ab89839101334ed1522'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:19.558456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.558745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.559038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.561347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 888/1761] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:20:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:20:19.563108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:19.575684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.585197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 874/1762] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:19 2022] DELETE /identity/v3/users/7300fbec15694ab89839101334ed1522 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:19.586265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:19.592194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.699786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541672-9b47-442b-909a-fc09e5d897ce admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:19.708668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:19.709771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.710199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.710487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.713440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066b50e-7d35-4c2f-9d22-1547e58e37f3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:19.715316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.833827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:19.834819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.835295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.835720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.846467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:19.891402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:19.894142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:19.916692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 875/1763] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:19.919633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:19.933074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c99e98-a853-44c8-bf6a-df8f6e89da9c service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:19.943172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:19.943876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:19.944168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:19.944484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:19.945381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:19.945693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:19.947617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:19.997833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 876/1764] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:19.999615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.001260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d3a99-f7d8-4424-b83d-9a3a937be3f0 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.011618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:20.024738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 889/1765] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:20.026085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.027067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.027252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.040228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3665333-f199-4de5-8f0a-ca1e4effebbe admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:20.064506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:20.070261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:20.070471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.070471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.070637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:20.070728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.070728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.080276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.104011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:20.192981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 877/1766] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:20 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:20.195317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:20.276242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 890/1767] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:20.375899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:20.425703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e96fbe-a570-4403-8d28-db3949226224 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:20.444079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:20.444905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.445488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.445843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.446977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.452900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.511453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 878/1768] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:20 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:20.516425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:20.533816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d861628-e454-4fcc-8a95-e0e00bc567fc None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:20.551041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:20.552082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.552601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.553074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.554036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.558765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.602736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 891/1769] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:20.604954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:20.622694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213429d-6166-4413-997c-4b0e3646af9b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:20.639880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:20.642683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.643782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.644745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.649243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.656402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7123ed9-7230-451d-b05b-01fa073a946b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.692453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 879/1770] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:20.696810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:20.716691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f7f19-9772-40c5-bba2-de67ede156ff admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:20.733539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:20.737716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.738631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.740231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] PATH_INFO: `/v3/projects/39e3a964023c4d51a9f8e721a6d83e7a/users/b1161310aa9a4195b326964dc96e19fb/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.789090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39e3a964023c4d51a9f8e721a6d83e7a, user_id=b1161310aa9a4195b326964dc96e19fb, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:20.795047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:20.830577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 892/1771] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:20 2022] PUT /identity/v3/projects/39e3a964023c4d51a9f8e721a6d83e7a/users/b1161310aa9a4195b326964dc96e19fb/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:20.841497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:20.842039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:20.842523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:20.989353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] MFA Rules not processed for user `b1161310aa9a4195b326964dc96e19fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:20.990611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:21.181785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:21.184132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:21.229729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 880/1772] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:21.236064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:21.362410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3052b91-9da8-4d04-8e60-3e622c8e560d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:21.393049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:21.394007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:21.394448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:21.394865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:21.399913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5573fe-1f87-4e19-a782-1157c478a808 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:21.426298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:21.427017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:21.428041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:21.430474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:21.431042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:21.432792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:21.435000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:21.443857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:21.480748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 893/1773] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:21.481723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:21.485168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:21.528637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 881/1774] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:21 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:21.533908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:21.544622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba8819e-ef35-4022-81fd-a8ac1d02be5e service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:21.705850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:21.879602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:21.881106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:21.884142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:21.884631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] PATH_INFO: `/v3/projects/439d0ca051354554bc86127f029ebddd` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:21.886539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ca42c-f9e5-4e2a-a748-80a8690ae0c7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:21.892952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=439d0ca051354554bc86127f029ebddd)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:21.898491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:21.902618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:21.935743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:21.942608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:21.942968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:21.943292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:21.949055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:21.955119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:22.022664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 882/1775] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:21 2022] POST /identity/v3/users => generated 300 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:22.031569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:22.070742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '439d0ca051354554bc86127f029ebddd', 'request_id': 'req-cad5ca33-062e-412e-af66-357258ac1e65'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:22.076182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46077c40-bb8c-4eb2-9530-e1ad4f275503 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:22.088438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '439d0ca051354554bc86127f029ebddd', 'request_id': 'req-cad5ca33-062e-412e-af66-357258ac1e65'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:22.106810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '439d0ca051354554bc86127f029ebddd', 'request_id': 'req-cad5ca33-062e-412e-af66-357258ac1e65'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:22.136748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] The token cache is being invalidate because project 439d0ca051354554bc86127f029ebddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:22.137612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:22.141140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-cad5ca33-062e-412e-af66-357258ac1e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:22.144179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 894/1776] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:21 2022] DELETE /identity/v3/projects/439d0ca051354554bc86127f029ebddd => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:22.152607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9e55-cea9-43ef-a487-77e092842e79 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.152899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9e55-cea9-43ef-a487-77e092842e79 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.153181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac9e55-cea9-43ef-a487-77e092842e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.209267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0fac9e55-cea9-43ef-a487-77e092842e79 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:22.218501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0fac9e55-cea9-43ef-a487-77e092842e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:22.318021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:22.318891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.319273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.319841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.320551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:22.322317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:22.344009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 883/1777] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:22.348089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:22.350131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:22.383489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4da2-1f16-4101-8a24-733310d94c90 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:22.404588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:22.407569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.407569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.407569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] PATH_INFO: `/v3/projects/643b5f7f978d42a19cc59447d2ab37ca/users/eedb963d6af74146b330a5cb4a350529/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.454305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643b5f7f978d42a19cc59447d2ab37ca, user_id=eedb963d6af74146b330a5cb4a350529, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:22.457965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:22.478484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 895/1778] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:22.489323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.489323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.489323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.494364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 884/1779] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:22 2022] PUT /identity/v3/projects/643b5f7f978d42a19cc59447d2ab37ca/users/eedb963d6af74146b330a5cb4a350529/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:22.508263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:22.563857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:22.569155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:22.684473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e07f6e5-dca0-4f67-923d-f4058e87ef41 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:22.692600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:22.693292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.693572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.693846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.694733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:22.698797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:22.721528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 885/1780] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:22 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:22.727269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:22.746606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546405f9-3c14-4de3-8935-8255fedb8370 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:22.768432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:22.769560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.771043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.774675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] PATH_INFO: `/v3/projects/643b5f7f978d42a19cc59447d2ab37ca/users/eedb963d6af74146b330a5cb4a350529/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.781255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643b5f7f978d42a19cc59447d2ab37ca, user_id=eedb963d6af74146b330a5cb4a350529, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:22.797876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34377678-9ace-48d5-b674-9f62b7ac0644 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:22.799215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 896/1781] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:22.805836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:22.834982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fcd69-369b-4c20-8ad7-b09a03db561d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:22.852342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 886/1782] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:22 2022] PUT /identity/v3/projects/643b5f7f978d42a19cc59447d2ab37ca/users/eedb963d6af74146b330a5cb4a350529/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:22.864151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:22.864466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:22.864801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:22.951046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:22.960998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] MFA Rules not processed for user `eedb963d6af74146b330a5cb4a350529`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:22.963232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:23.001744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:23.040137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.040797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.041077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.041603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.042881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:23.043087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.044775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.080611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 897/1783] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:23.084678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.099535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa44d2-358a-4d86-bc27-f00f99aa9eba admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.105675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.106405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.106989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.107378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.108765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.110858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.132455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 887/1784] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:23.142400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.143963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:23.150271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 898/1785] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:23 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:23.152258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.233204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcfb247-6846-4b45-9771-95fa84ed45cc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.236982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b227c62-7cff-4bf4-9d20-e17d40634d62 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.248311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.249564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.250007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.250949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.252068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.257423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.304555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 899/1786] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:23 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:23.308142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.321926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1c7c0-cce1-44fa-8b23-4ae62f8dc7cf admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.327557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.329418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.329712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.331198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.333700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.334766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.335106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.335653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.336975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.342992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.361338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 900/1787] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:23.362375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.378627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e9b81-b60d-4d2b-8581-6b643df9cea8 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.384414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.385671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.386123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.386603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.387514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.389271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.408344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 901/1788] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:23.412492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.426308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22068905-10a8-43bf-981e-79d9d06b9860 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.443885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.444808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.445907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.446192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] PATH_INFO: `/v3/projects/ed33c59bbad3461db47d510192b3d4a7/users/9d3c2de007a6409daa4b0e171eb0686c/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.487723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed33c59bbad3461db47d510192b3d4a7, user_id=9d3c2de007a6409daa4b0e171eb0686c, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.494368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33d623-a417-488c-9c2c-0cbf1f1bb970 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.519553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.521084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.538107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 902/1789] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:23 2022] PUT /identity/v3/projects/ed33c59bbad3461db47d510192b3d4a7/users/9d3c2de007a6409daa4b0e171eb0686c/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:23.547637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 888/1790] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:23.547959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282c607-34f0-49fd-9b91-68b14c5cca11 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.547959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282c607-34f0-49fd-9b91-68b14c5cca11 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.547959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282c607-34f0-49fd-9b91-68b14c5cca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.559390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.572213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:23.594383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b1448-59be-47d0-873b-42361e412eb2 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:23.618385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d282c607-34f0-49fd-9b91-68b14c5cca11 None None] MFA Rules not processed for user `2befe74ad06542b785d326101fc06d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:23.622230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d282c607-34f0-49fd-9b91-68b14c5cca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:23.650839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:23.780874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:23.781245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:23.782479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.782854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.783020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] PATH_INFO: `/v3/projects/c2b0f66ad6bb4ece98ea1859904d2a11` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.787287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2b0f66ad6bb4ece98ea1859904d2a11)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:23.789297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:23.820221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 903/1791] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:23.821720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40fb5d-ffa0-4692-a7d7-d60cbdd67fb8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:23.822014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40fb5d-ffa0-4692-a7d7-d60cbdd67fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:23.822292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40fb5d-ffa0-4692-a7d7-d60cbdd67fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:23.894721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7e40fb5d-ffa0-4692-a7d7-d60cbdd67fb8 None None] MFA Rules not processed for user `9d3c2de007a6409daa4b0e171eb0686c`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:23.898189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7e40fb5d-ffa0-4692-a7d7-d60cbdd67fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:23.906518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2b0f66ad6bb4ece98ea1859904d2a11', 'request_id': 'req-46d97de5-6624-441e-ae82-e4f608bf10de'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:23.915528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2b0f66ad6bb4ece98ea1859904d2a11', 'request_id': 'req-46d97de5-6624-441e-ae82-e4f608bf10de'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:23.931261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2b0f66ad6bb4ece98ea1859904d2a11', 'request_id': 'req-46d97de5-6624-441e-ae82-e4f608bf10de'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:23.954500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] The token cache is being invalidate because project c2b0f66ad6bb4ece98ea1859904d2a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:23.954788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:23.955603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:23.964736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 889/1792] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:23 2022] DELETE /identity/v3/projects/c2b0f66ad6bb4ece98ea1859904d2a11 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:23.964736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:23.967704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.065339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 904/1793] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:24.065339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.065339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.065339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.094548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:24.099137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.173801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d97de5-6624-441e-ae82-e4f608bf10de admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.187547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.187833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.188112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.188404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] PATH_INFO: `/v3/users/b1161310aa9a4195b326964dc96e19fb` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.195175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1161310aa9a4195b326964dc96e19fb)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.197342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.228728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 905/1794] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:24.228728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.228728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.249398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a2b96-7576-4d11-8ccd-b238e25ca0f7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.263078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.264564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.264849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.265126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.265891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.268503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d69076-ce45-45da-8885-a773c7306709 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.320708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 906/1795] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:24.324588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1161310aa9a4195b326964dc96e19fb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:24.325343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1161310aa9a4195b326964dc96e19fb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:24.325998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1161310aa9a4195b326964dc96e19fb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:24.326185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfb0fa-b569-4d40-9336-591528f1aa6e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.327371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfb0fa-b569-4d40-9336-591528f1aa6e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.329980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:24.337883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfb0fa-b569-4d40-9336-591528f1aa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.342227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1161310aa9a4195b326964dc96e19fb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:24.373482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d825e8c9-a82a-4102-ab10-860575a95fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1161310aa9a4195b326964dc96e19fb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:24.411383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 890/1796] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:23 2022] DELETE /identity/v3/users/b1161310aa9a4195b326964dc96e19fb => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:24.412892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.415272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.415638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.419996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 891/1797] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:20:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:20:24.420871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.427924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.437539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d0dfb0fa-b569-4d40-9336-591528f1aa6e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:24.449119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d0dfb0fa-b569-4d40-9336-591528f1aa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.554381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:24.564478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 907/1798] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:24.566654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baaaaa-4381-49b1-9377-db18a6c28bf0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.566878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baaaaa-4381-49b1-9377-db18a6c28bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.567504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83baaaaa-4381-49b1-9377-db18a6c28bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.569864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef76971-3b8a-4fde-bb20-342b2968b8fa None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.582583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.583289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.583571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.583842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.584566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.586885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.597031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-83baaaaa-4381-49b1-9377-db18a6c28bf0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:24.598076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-83baaaaa-4381-49b1-9377-db18a6c28bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.612534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 908/1799] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:24.614201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7bb6-e6ba-49d2-bd47-08a6ea5ee76a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.614723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7bb6-e6ba-49d2-bd47-08a6ea5ee76a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.615498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f7bb6-e6ba-49d2-bd47-08a6ea5ee76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.640671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 892/1800] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:24.641662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.656970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-534f7bb6-e6ba-49d2-bd47-08a6ea5ee76a None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:24.657909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ea78a-997d-4f02-9492-c6bfa2fdd137 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.658777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-534f7bb6-e6ba-49d2-bd47-08a6ea5ee76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.672887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.673572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.673852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.674130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.675045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:24.675803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.677750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.705245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 893/1801] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:24.709660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.723186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570434-e6f6-4a0d-a732-e653b1e6d63b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.730920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.731761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.732495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.733083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.733966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.736221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.753543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 894/1802] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:24.754776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.756454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.816382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 909/1803] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:24.818630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d161-d4bb-4f72-96ce-9129fec382b9 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.819235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d161-d4bb-4f72-96ce-9129fec382b9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.819693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d161-d4bb-4f72-96ce-9129fec382b9 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.829078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 910/1804] 10.222.0.45 () {54 vars in 912 bytes} [Tue May 24 15:20:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:20:24.829788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d161-d4bb-4f72-96ce-9129fec382b9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.854491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d161-d4bb-4f72-96ce-9129fec382b9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.864182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744292d-24e9-49c1-a3cf-a3c4cd8a1c26 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.867673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.868531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.869932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.870337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.871139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.871542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.872533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.872929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.873672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.875365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82929a84-a383-4fd1-9457-7e2a8a63ece7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.891482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.893256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.910873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 895/1805] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:24.910873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:24.925544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12f115-e923-42e6-b2bd-ed93d8fee835 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:24.933787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:24.935627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.935954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.936279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] PATH_INFO: `/v3/projects/77cf235e7fb2422791ea98b8f511dd1e/users/35d18baa784744ee865689fb3ea80860/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.941804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 911/1806] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:24.944706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:24.945012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:24.945297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:24.968196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cf235e7fb2422791ea98b8f511dd1e, user_id=35d18baa784744ee865689fb3ea80860, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:24.971186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:24.972106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:24.972571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:24.998373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 896/1807] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:24 2022] PUT /identity/v3/projects/77cf235e7fb2422791ea98b8f511dd1e/users/35d18baa784744ee865689fb3ea80860/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:24.999535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.019025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:25.057930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 912/1808] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:25.058900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.108483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf86ff-3cc1-4b99-889b-3ef74b379e36 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.116513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.118305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.118654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.118937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.119703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.122228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.152821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4d79ab-6071-4c0e-89fe-a363eeaf9116 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.158641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.159617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.160407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.160827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.161721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.163604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.169721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 897/1809] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:24 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:25.171010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.173208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:25.180712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 913/1810] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:25.180712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.187373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:25.210247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:25.301150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb06f50-ac5f-47b8-99f8-f764d94f13cd admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.305447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.306013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aac12-69dd-45c6-b643-1d18fec28202 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.306129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.306393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.306671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.313262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.314152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.314380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.314590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.322160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.324489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.327837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.330953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.342715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 914/1811] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:25.343751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.344533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 898/1812] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:25.345435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.357456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7eabd8-7370-433d-aa03-52999623973d service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.363378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.364344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.364735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.364959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] PATH_INFO: `/v3/projects/77cf235e7fb2422791ea98b8f511dd1e/users/35d18baa784744ee865689fb3ea80860/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.367037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=77cf235e7fb2422791ea98b8f511dd1e, user_id=35d18baa784744ee865689fb3ea80860, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.368764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ca8d9-bad7-4c67-b072-8d52eadaab28 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.370995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a636053-6003-47cd-bb7a-98351e043b7d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.392916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 899/1813] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:25 2022] PUT /identity/v3/projects/77cf235e7fb2422791ea98b8f511dd1e/users/35d18baa784744ee865689fb3ea80860/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:25.401657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.402475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.403502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.404648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.404938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.405318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.416564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.417607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.422042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.436857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] MFA Rules not processed for user `35d18baa784744ee865689fb3ea80860`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:25.438435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:25.460917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 915/1814] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:25.465317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.492255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954d696-9062-4363-b3de-a19ca78f0b4c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.574688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:25.587353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 900/1815] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:25.599570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.601733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:25.639836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.640658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.640936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.641210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.642365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.643533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.667624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 916/1816] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:25.670078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:25.689303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa545a-743f-4946-9d0a-088f5a7ec215 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.715786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.717314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.721860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.722289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] PATH_INFO: `/v3/projects/dd36869874a04759baf422592c2c6f2a/users/216de7b154fc41aea86fb8c1c3be7636/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.738032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1b33-7f72-4ecf-b42e-19e5204a127e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:25.745191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:25.745859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.746137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.746406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.756501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd36869874a04759baf422592c2c6f2a, user_id=216de7b154fc41aea86fb8c1c3be7636, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.759396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25fa65-8bf2-4b3b-9498-742db3232cb8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.772984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:25.774681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:25.797098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 917/1817] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:25 2022] PUT /identity/v3/projects/dd36869874a04759baf422592c2c6f2a/users/216de7b154fc41aea86fb8c1c3be7636/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:25.814945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:25.815238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:25.815513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:25.820089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 901/1818] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:25.893956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] MFA Rules not processed for user `216de7b154fc41aea86fb8c1c3be7636`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:25.895328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:25.987065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 918/1819] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:25.988684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.012535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0730-f00e-4554-abff-db72e3fd91e1 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:26.141511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:26.142296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.145034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.145034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.145473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:26.147976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:26.207878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 902/1820] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:25 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:26.212214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.225720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:26.235324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec21a2-1f40-4c86-a7a5-e0a1297ff7e0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:26.376583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.379744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:26.478262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:26.478262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.478262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.483466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.484504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:26.488858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:26.555785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 919/1821] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:26 2022] POST /identity/v3/users => generated 297 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:26.555785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.582869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229790-30dc-4624-a9b2-528f27f49d9e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:26.586329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca37bd-8d04-459f-82d8-86090fa36c57 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:26.600045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:26.601288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.601288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.601288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.602048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:26.603668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:26.603859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:26.607554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.608815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.609207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] PATH_INFO: `/v3/projects/39e3a964023c4d51a9f8e721a6d83e7a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.613310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] RBAC: Authorizing `identity:delete_project(project_id=39e3a964023c4d51a9f8e721a6d83e7a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:26.621320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:26.652644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 920/1822] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:26.653577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.670448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9135-973f-4ed2-8a03-c744e1986f5c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:26.683370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:26.684867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.685846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.685846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] PATH_INFO: `/v3/projects/54cea79ae8dd4a4797853409445c9a36/users/a77ddbb648c6490faf0ea9ea4857c4e7/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.706646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cea79ae8dd4a4797853409445c9a36, user_id=a77ddbb648c6490faf0ea9ea4857c4e7, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:26.711581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db2475-5173-4f25-8d00-8c1eafc1f8b8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:26.741665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 921/1823] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:26 2022] PUT /identity/v3/projects/54cea79ae8dd4a4797853409445c9a36/users/a77ddbb648c6490faf0ea9ea4857c4e7/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:26.742952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:26.746136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:26.747219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:26.748339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:26.773357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] MFA Rules not processed for user `40db5795ce7e4c5bbf06fb81ce404e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:26.773833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:26.777594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39e3a964023c4d51a9f8e721a6d83e7a', 'request_id': 'req-cc29a077-2456-4322-8b6b-a00c4944053c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:26.791049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39e3a964023c4d51a9f8e721a6d83e7a', 'request_id': 'req-cc29a077-2456-4322-8b6b-a00c4944053c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:26.809113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39e3a964023c4d51a9f8e721a6d83e7a', 'request_id': 'req-cc29a077-2456-4322-8b6b-a00c4944053c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:26.832763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] The token cache is being invalidate because project 39e3a964023c4d51a9f8e721a6d83e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:26.833457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:26.834665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:26.841006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 903/1824] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:26 2022] DELETE /identity/v3/projects/39e3a964023c4d51a9f8e721a6d83e7a => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:26.842137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.857559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:26.904631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 922/1825] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:26.906276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:26.909114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:27.017163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29a077-2456-4322-8b6b-a00c4944053c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:27.024010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:27.025368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:27.030594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:27.030945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:27.031996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:27.034366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:27.046875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 904/1826] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:27.051774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:27.064810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fefe6b-0947-40f9-a5de-8ad141300cbe admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:27.071939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd287dd1-eaeb-4786-83ca-150bfebed810 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:27.080068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:27.083021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:27.083687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:27.084238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] PATH_INFO: `/v3/projects/54cea79ae8dd4a4797853409445c9a36/users/a77ddbb648c6490faf0ea9ea4857c4e7/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:27.089192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:27.090103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:27.090416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:27.090665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] PATH_INFO: `/v3/users/eedb963d6af74146b330a5cb4a350529` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:27.093454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54cea79ae8dd4a4797853409445c9a36, user_id=a77ddbb648c6490faf0ea9ea4857c4e7, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:27.102707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819fbf5-f583-4a40-87c8-35729d0ecfd1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:27.105255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eedb963d6af74146b330a5cb4a350529)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:27.107235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:27.113249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:27.138563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 905/1827] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:27 2022] PUT /identity/v3/projects/54cea79ae8dd4a4797853409445c9a36/users/a77ddbb648c6490faf0ea9ea4857c4e7/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:27.144329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:27.144329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:27.144329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:27.200597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] MFA Rules not processed for user `a77ddbb648c6490faf0ea9ea4857c4e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:27.201847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:27.220269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eedb963d6af74146b330a5cb4a350529'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:27.220998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eedb963d6af74146b330a5cb4a350529'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:27.222830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eedb963d6af74146b330a5cb4a350529'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:27.240822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eedb963d6af74146b330a5cb4a350529'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:27.249702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eedb963d6af74146b330a5cb4a350529'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:27.290575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 923/1828] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:26 2022] DELETE /identity/v3/users/eedb963d6af74146b330a5cb4a350529 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:27.368841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 906/1829] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:27.378990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:27.380647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:27.468807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687edd-8e57-4c13-bd7d-3a7fb5ed81c0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:27.483571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:27.484341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:27.484623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:27.484897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:27.602777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:27.606351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d0eada-2863-4a69-9e4e-e8edb6d27fdf service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:27.638529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 924/1830] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:27.763830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:27.765466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:27.821586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:27.919439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809becd1-5991-41ae-bd39-405da1f6aeea None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:28.095136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:28.097295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.106470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.107795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] PATH_INFO: `/v3/projects/643b5f7f978d42a19cc59447d2ab37ca` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.112796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643b5f7f978d42a19cc59447d2ab37ca)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:28.116186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:28.314324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643b5f7f978d42a19cc59447d2ab37ca', 'request_id': 'req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.347260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643b5f7f978d42a19cc59447d2ab37ca', 'request_id': 'req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.366445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643b5f7f978d42a19cc59447d2ab37ca', 'request_id': 'req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.394854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] The token cache is being invalidate because project 643b5f7f978d42a19cc59447d2ab37ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:28.395153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.397649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.400539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 907/1831] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:27 2022] DELETE /identity/v3/projects/643b5f7f978d42a19cc59447d2ab37ca => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:28.405636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328b8a-fc0f-4895-ad09-11d3bae3c4ce None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.406091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328b8a-fc0f-4895-ad09-11d3bae3c4ce None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.406640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56328b8a-fc0f-4895-ad09-11d3bae3c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.438362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:28.440629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:28.462945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-56328b8a-fc0f-4895-ad09-11d3bae3c4ce None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:28.471021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-56328b8a-fc0f-4895-ad09-11d3bae3c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:28.562445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:28.605223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642cf6-d7c2-4ef5-96f2-99313f721fe7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:28.610906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77ddbb648c6490faf0ea9ea4857c4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54cea79ae8dd4a4797853409445c9a36', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:28.611582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.611867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.612144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.623328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:28.623328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:28.623328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:28.625585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:28.649592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 908/1832] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:20:28.651816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:28.669034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 925/1833] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:28.678549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.678863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.679199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.683583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17c674-1a2f-4054-990c-b7360349a707 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:28.699764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77ddbb648c6490faf0ea9ea4857c4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54cea79ae8dd4a4797853409445c9a36', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:28.700488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.700773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.701428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.701876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:28.702468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:28.703700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:28.703806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:28.712225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 926/1834] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:28.723403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:28.725425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:28.738762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cab7888-3fbf-45e4-8af7-480ade1790dd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:28.744044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:28.745015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:28.745336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:28.745626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] PATH_INFO: `/v3/users/9d3c2de007a6409daa4b0e171eb0686c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:28.746870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d3c2de007a6409daa4b0e171eb0686c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:28.749379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:28.798085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 909/1835] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:28 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:28.798085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:28.821881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d3c2de007a6409daa4b0e171eb0686c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.822707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d3c2de007a6409daa4b0e171eb0686c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.824114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d3c2de007a6409daa4b0e171eb0686c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.843856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d3c2de007a6409daa4b0e171eb0686c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.868465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d3c2de007a6409daa4b0e171eb0686c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:28.906359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 927/1836] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:28 2022] DELETE /identity/v3/users/9d3c2de007a6409daa4b0e171eb0686c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:28.907290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:28.908712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:28.914399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96dc4a-69ad-4a55-9ba3-ebf8bd584f8c tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.037851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.038541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.038821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.039097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.039834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:29.042793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.044533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2258b9e-bbf3-4069-83ee-6adfd571ed85 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.050848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab3bc01-34c9-4736-a72a-737c67f40bfb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.073710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77ddbb648c6490faf0ea9ea4857c4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54cea79ae8dd4a4797853409445c9a36', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.076438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.076438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.076438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.081037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:29.081363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.085408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.087090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 910/1837] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:29.089689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.089983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.090261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.120210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 928/1838] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:29.120210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.121689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.131502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:29.132733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.144263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f95460-f0bb-420b-9c37-8be3edb7f1dc tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.150443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 911/1839] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:29.154363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.154813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.155077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.155352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.156055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.158187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.198412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 929/1840] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:29 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:29.202315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.215270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7de5e-b2e0-4521-811a-6cc035826c44 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.223645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.242079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:29.374848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.386732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.389913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.390530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.393433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.396064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.397220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d831b-679e-4b9b-a891-48aee462bb96 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.410385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a77ddbb648c6490faf0ea9ea4857c4e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '54cea79ae8dd4a4797853409445c9a36', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.411756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.411756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.412275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] PATH_INFO: `/v3/projects/b539d6f9e55d4b5a90c93e76e4b4cfa7` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.415398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b539d6f9e55d4b5a90c93e76e4b4cfa7)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.419432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.437922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:29.444835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 912/1841] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:29 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:29.446040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.452611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.536357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b539d6f9e55d4b5a90c93e76e4b4cfa7', 'request_id': 'req-d678db98-39f0-439b-826d-972713832805'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:29.551042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b539d6f9e55d4b5a90c93e76e4b4cfa7', 'request_id': 'req-d678db98-39f0-439b-826d-972713832805'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:29.569729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b539d6f9e55d4b5a90c93e76e4b4cfa7', 'request_id': 'req-d678db98-39f0-439b-826d-972713832805'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:29.588437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] The token cache is being invalidate because project b539d6f9e55d4b5a90c93e76e4b4cfa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:29.588877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:29.592169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:29.594662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 930/1842] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:29 2022] DELETE /identity/v3/projects/b539d6f9e55d4b5a90c93e76e4b4cfa7 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:29.595502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.597054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.599132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64edf36-d071-495d-b9e7-816884f25919 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.705952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.707956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.709266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.709377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.714758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.748564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678db98-39f0-439b-826d-972713832805 tempest-VolumeQuotaClassesTest-1129688077 tempest-VolumeQuotaClassesTest-1129688077-project-admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.751494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.753150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.763099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.763736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.764131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.764532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.766711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.769759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6fcf6-bb43-4856-9f14-0ab38853a762 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.774822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 913/1843] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:29.793422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 931/1844] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:29.795683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.812697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623ac2b-e4e1-4f86-a62e-a749faceb665 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.829149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.830186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.830571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.831654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.833514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.835608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cceb1-6c5d-4b92-a383-5e277cbeeda6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:29.848486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa04d-f46a-4fe5-99e4-b3ddfcb32fe0 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.849151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa04d-f46a-4fe5-99e4-b3ddfcb32fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.849595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafa04d-f46a-4fe5-99e4-b3ddfcb32fe0 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.851243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 932/1845] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 15:20:29.856339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 914/1846] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:29.860188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa04d-f46a-4fe5-99e4-b3ddfcb32fe0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:29.875050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.876233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.876758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.888989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafa04d-f46a-4fe5-99e4-b3ddfcb32fe0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:29.900780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:29.903027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:29.909264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:29.910373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:29.910639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:29.910920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] PATH_INFO: `/v3/projects/ee231ae88759462284eb29a20172e8df/users/9ae765fde07a4c4d9120c99cf358f179/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:29.955155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee231ae88759462284eb29a20172e8df, user_id=9ae765fde07a4c4d9120c99cf358f179, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:29.974074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdc614-122f-45c3-b22d-09fc72dc5b04 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.010080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 933/1847] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:29 2022] PUT /identity/v3/projects/ee231ae88759462284eb29a20172e8df/users/9ae765fde07a4c4d9120c99cf358f179/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:30.018168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 915/1848] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:20:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:30.021175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.021468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.021747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.039907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.046180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:30.047298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.058997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] MFA Rules not processed for user `9ae765fde07a4c4d9120c99cf358f179`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:30.062143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.094026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb08506-1800-40c7-a6c9-221d1b2893fd None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.149921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:30.182598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 934/1849] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:30.182598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.182865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.184028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.185588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.186068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.186536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] PATH_INFO: `/v3/users/216de7b154fc41aea86fb8c1c3be7636` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.188042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=216de7b154fc41aea86fb8c1c3be7636)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.190591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.290403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '216de7b154fc41aea86fb8c1c3be7636'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:30.292038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '216de7b154fc41aea86fb8c1c3be7636'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:30.293239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '216de7b154fc41aea86fb8c1c3be7636'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:30.335866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '216de7b154fc41aea86fb8c1c3be7636'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:30.354009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18ba78-dbc0-4aef-97a4-1f007b4e6efe None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.372673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.373347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.373674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.373954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.376269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '216de7b154fc41aea86fb8c1c3be7636'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:30.378877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.403082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 916/1850] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:30 2022] DELETE /identity/v3/users/216de7b154fc41aea86fb8c1c3be7636 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:30.404183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.407647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.415652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.417346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d46378-1e02-4605-a606-a92c13d9e20f service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.451646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 935/1851] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:30.458836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.458836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.459129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.513577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:30.518223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.552381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 936/1852] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:30.552509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef70803-299c-4f50-86b1-537b9779ae00 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.566039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.566732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.567015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.567287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.567983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.573757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.623236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 917/1853] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:30 2022] POST /identity/v3/projects => generated 389 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:30.627515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.645879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760d413-c64c-4b0a-8dd2-04aa9dc5e464 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.733650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.734687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.735148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.735453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.736906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.740531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.768706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.772091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:30.803516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 937/1854] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:30 2022] POST /identity/v3/users => generated 301 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:30.807184 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.831468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ea6a-9ce7-4bd9-9953-22fd43e0e6a8 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.841060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.842964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.843555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.845030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.845722 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.856278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.863397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:30.880444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 938/1855] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:30.889463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:30.907554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76a5b9-fae0-450b-9396-0c08b98bb7ab admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:30.920057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:30.921806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:30.921806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:30.921806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] PATH_INFO: `/v3/projects/bcf646f22cc6410aac94eb976f3fe05e/users/38ee9b445c6f4a9aa96568883e643272/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:30.953647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:30.962527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf646f22cc6410aac94eb976f3fe05e, user_id=38ee9b445c6f4a9aa96568883e643272, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:30.967438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:30.997094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 939/1856] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:30 2022] PUT /identity/v3/projects/bcf646f22cc6410aac94eb976f3fe05e/users/38ee9b445c6f4a9aa96568883e643272/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:31.000932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.003688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.005672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b2843-abcf-4e84-b1d4-351b5bd99472 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.136739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.137547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.137843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.138114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.167941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8fcb3-cd1f-4e09-a593-55135c5aa374 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.173550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.174421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.174762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.175030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] PATH_INFO: `/v3/projects/ed33c59bbad3461db47d510192b3d4a7` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.176288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed33c59bbad3461db47d510192b3d4a7)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.178555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.205969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.207674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.222060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 918/1857] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:31.223051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.236698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebec0d1-7ab9-4d1e-a8a4-c086e2113f93 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.241105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed33c59bbad3461db47d510192b3d4a7', 'request_id': 'req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.249926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed33c59bbad3461db47d510192b3d4a7', 'request_id': 'req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.281478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed33c59bbad3461db47d510192b3d4a7', 'request_id': 'req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.310125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] The token cache is being invalidate because project ed33c59bbad3461db47d510192b3d4a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:31.310335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.311501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.313831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 940/1858] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:30 2022] DELETE /identity/v3/projects/ed33c59bbad3461db47d510192b3d4a7 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:31.325393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.325393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.340649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.342895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.343181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.343605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.344537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.346190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.376384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 919/1859] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:31.380046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.383169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.406958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b7e09-f2d8-4e62-a53d-ffeea02744c9 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.412531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.413494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.413774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.414042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] PATH_INFO: `/v3/projects/bcf646f22cc6410aac94eb976f3fe05e/users/38ee9b445c6f4a9aa96568883e643272/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.417662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf646f22cc6410aac94eb976f3fe05e, user_id=38ee9b445c6f4a9aa96568883e643272, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.420750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea43b9-b656-4790-967b-95262695b06e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.436285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 920/1860] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:31 2022] PUT /identity/v3/projects/bcf646f22cc6410aac94eb976f3fe05e/users/38ee9b445c6f4a9aa96568883e643272/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:31.442328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a4433-cd98-45f9-ae9d-a0dcffafeaf9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.447858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.448191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.448511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.471657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:31.500785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] MFA Rules not processed for user `38ee9b445c6f4a9aa96568883e643272`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:31.502838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.529611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:31.582885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.584342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.584769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.585201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] PATH_INFO: `/v3/users/35d18baa784744ee865689fb3ea80860` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.591132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d18baa784744ee865689fb3ea80860)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.593628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.614264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 921/1861] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:31.628827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.630692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.690135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d18baa784744ee865689fb3ea80860'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.691247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d18baa784744ee865689fb3ea80860'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.692087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d18baa784744ee865689fb3ea80860'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.714195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d18baa784744ee865689fb3ea80860'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.728427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d18baa784744ee865689fb3ea80860'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:31.730591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b24e9-ac3e-4ee1-8d9f-b02e2d5df76a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:31.742970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:31.743648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:31.743928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:31.744202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:31.752852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.755818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 941/1862] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:31 2022] DELETE /identity/v3/users/35d18baa784744ee865689fb3ea80860 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:31.768715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:31.770854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:31.804123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 922/1863] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:31.943973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.945659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.955005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:31.956584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:31.975884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9c604-5bfa-4483-b243-38fe4b1377e6 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:32.084998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496476-81db-4693-b0d0-5e2a90e5baa9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:32.098103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:32.098103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:32.098368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:32.098709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] PATH_INFO: `/v3/users/9ae765fde07a4c4d9120c99cf358f179` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:32.105173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae765fde07a4c4d9120c99cf358f179)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:32.116621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:32.118139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:32.118605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:32.118704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:32.119079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] PATH_INFO: `/v3/projects/dd36869874a04759baf422592c2c6f2a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:32.121317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd36869874a04759baf422592c2c6f2a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:32.124982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:32.219427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:32.258069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd36869874a04759baf422592c2c6f2a', 'request_id': 'req-7f814f5c-46f3-4aa6-80ab-e5cfed174734'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.263196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae765fde07a4c4d9120c99cf358f179'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.266125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae765fde07a4c4d9120c99cf358f179'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.266724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae765fde07a4c4d9120c99cf358f179'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.273708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd36869874a04759baf422592c2c6f2a', 'request_id': 'req-7f814f5c-46f3-4aa6-80ab-e5cfed174734'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.277620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:32.283446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae765fde07a4c4d9120c99cf358f179'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.292965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd36869874a04759baf422592c2c6f2a', 'request_id': 'req-7f814f5c-46f3-4aa6-80ab-e5cfed174734'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.300577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bba1ea12-b54c-4df2-983a-bcf7db4c1207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae765fde07a4c4d9120c99cf358f179'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.312243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] The token cache is being invalidate because project dd36869874a04759baf422592c2c6f2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:32.312571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.313642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.319406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 923/1864] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:31 2022] DELETE /identity/v3/projects/dd36869874a04759baf422592c2c6f2a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:32.328176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:32.331299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:32.354622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 942/1865] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:31 2022] DELETE /identity/v3/users/9ae765fde07a4c4d9120c99cf358f179 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:32.361186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:32.361471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:32.361750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:32.394014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:32.396901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:32.519295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 943/1866] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:32.613374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814f5c-46f3-4aa6-80ab-e5cfed174734 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:32.622566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:32.624705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:32.625228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:32.625751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] PATH_INFO: `/v3/users/a77ddbb648c6490faf0ea9ea4857c4e7` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:32.629154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a77ddbb648c6490faf0ea9ea4857c4e7)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:32.633420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:32.638465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:32.643892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:32.861180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed25c08-545b-446c-b89b-012cd408fc45 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:32.881432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:32.882118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:32.882417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:32.882803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:32.913443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a77ddbb648c6490faf0ea9ea4857c4e7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.917131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a77ddbb648c6490faf0ea9ea4857c4e7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.919910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:32.920464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a77ddbb648c6490faf0ea9ea4857c4e7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:32.921516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:32.949586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 944/1867] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:32.979495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a77ddbb648c6490faf0ea9ea4857c4e7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.050543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-df10184b-bc8f-4006-a022-b0f7bfce3798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a77ddbb648c6490faf0ea9ea4857c4e7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.144377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 924/1868] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:32 2022] DELETE /identity/v3/users/a77ddbb648c6490faf0ea9ea4857c4e7 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:33.208757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:33.210231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:33.230789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357bf79-e421-489d-9ae4-e48a731fc744 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:33.361785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:33.402999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:33.404693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:33.405037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:33.405447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] PATH_INFO: `/v3/projects/77cf235e7fb2422791ea98b8f511dd1e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:33.409316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=77cf235e7fb2422791ea98b8f511dd1e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:33.412257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:33.550177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77cf235e7fb2422791ea98b8f511dd1e', 'request_id': 'req-f52c3aa4-c508-4d64-8917-3c4900246aef'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.559115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77cf235e7fb2422791ea98b8f511dd1e', 'request_id': 'req-f52c3aa4-c508-4d64-8917-3c4900246aef'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.572318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77cf235e7fb2422791ea98b8f511dd1e', 'request_id': 'req-f52c3aa4-c508-4d64-8917-3c4900246aef'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.594682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] The token cache is being invalidate because project 77cf235e7fb2422791ea98b8f511dd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:33.596119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.597211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f52c3aa4-c508-4d64-8917-3c4900246aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:33.600682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 945/1869] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:33 2022] DELETE /identity/v3/projects/77cf235e7fb2422791ea98b8f511dd1e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:33.606350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:33.606640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:33.606917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:33.637666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:33.641603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:33.806851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 925/1870] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:33.812912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:33.813200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:33.813484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:33.850733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:33.853636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:33.887402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:33.892063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 946/1871] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:33.912976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc69dd-54e4-4fdf-8275-06c74305e15c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:33.933353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:33.935864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:33.938281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:33.939313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:33.943075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:33.944584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:33.949557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:33.954540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:33.981933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 926/1872] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:33.986913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:33.997940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388011f-ef81-411c-bd14-2d2b9f702d39 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.003464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.004685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.005183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.006016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.006924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.008822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.010966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.021790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.031099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652142-ea33-4100-9dc7-e4a258a53e1e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.042913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.044108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.045107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.045700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] PATH_INFO: `/v3/projects/54cea79ae8dd4a4797853409445c9a36` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.047135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54cea79ae8dd4a4797853409445c9a36)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.049325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.066316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 947/1873] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:33 2022] POST /identity/v3/projects => generated 365 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:34.068686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.149346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54cea79ae8dd4a4797853409445c9a36', 'request_id': 'req-77d307b9-b606-4f9c-8759-f9df69912a27'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.165512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54cea79ae8dd4a4797853409445c9a36', 'request_id': 'req-77d307b9-b606-4f9c-8759-f9df69912a27'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.176890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54cea79ae8dd4a4797853409445c9a36', 'request_id': 'req-77d307b9-b606-4f9c-8759-f9df69912a27'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.196484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] The token cache is being invalidate because project 54cea79ae8dd4a4797853409445c9a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:34.198910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.199732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-77d307b9-b606-4f9c-8759-f9df69912a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.204487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 927/1874] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:34 2022] DELETE /identity/v3/projects/54cea79ae8dd4a4797853409445c9a36 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:34.212133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ec8e-25b8-4808-83d7-1e8e79d57598 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.212538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ec8e-25b8-4808-83d7-1e8e79d57598 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.212809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ec8e-25b8-4808-83d7-1e8e79d57598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.233832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff8312-1597-4aa6-abad-bc6373c739af admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.236661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7312ec8e-25b8-4808-83d7-1e8e79d57598 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:34.237498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7312ec8e-25b8-4808-83d7-1e8e79d57598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.319054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 928/1875] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:34.332509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.332509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.332509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.385041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:34.387718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.396454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.397237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.397589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.398006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.398841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.401839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.412578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:34.414344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 929/1876] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:34.415343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.435471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ecac8-7176-40ef-bca2-79da0ffeaea2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.436175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 948/1877] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/users => generated 283 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:34.440908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.442820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.451359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.452239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.452614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.452908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.453748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:34.454055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.455797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.458631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62d2f9-1729-4e55-8d81-f7ca827f0359 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.467589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.472965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.473173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.473539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.476385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.477877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.495447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 949/1879] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:34.498235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 930/1879] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:34.499329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.499462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.515479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0dd145-8cba-4280-84d8-570468cf83da admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.517193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72382076-6ca7-4c18-9623-6bf0451bea2a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.520380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.521064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.521348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.521621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.522351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.522758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.523532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.523817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.524086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.524292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.524820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.527482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.539554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 950/1880] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:34.543617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.559861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce59c4-c3a6-4fdd-b0ed-6e6287869ff7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.576896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 931/1881] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:34.578062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.581952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.585406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.586352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.586608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.588145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] PATH_INFO: `/v3/projects/7694a9dfbef5453db04f7973f32f3455/users/d0a7dd85292b40d7817c8e0ce9d956ca/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.643473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7694a9dfbef5453db04f7973f32f3455, user_id=d0a7dd85292b40d7817c8e0ce9d956ca, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.646368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.673311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 951/1882] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:34 2022] PUT /identity/v3/projects/7694a9dfbef5453db04f7973f32f3455/users/d0a7dd85292b40d7817c8e0ce9d956ca/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:34.674241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.694416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:34.754089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a91ef-cc4a-4727-bf68-1458f61e643a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.781665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784bd7a-b37f-46ea-b0e1-4b4e9e0d0b28 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:34.787834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.788733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.789015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.789291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.789997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.792151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0447b8-822d-496c-ad1f-086f0f842652 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.832071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 952/1883] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:34.834153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.834488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.834924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.844351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:34.845234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:34.845511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:34.845780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] PATH_INFO: `/v3/projects/ee231ae88759462284eb29a20172e8df` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:34.849064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee231ae88759462284eb29a20172e8df)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:34.851597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:34.884754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] MFA Rules not processed for user `d0a7dd85292b40d7817c8e0ce9d956ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:34.887723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.922042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee231ae88759462284eb29a20172e8df', 'request_id': 'req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.928642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee231ae88759462284eb29a20172e8df', 'request_id': 'req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.940252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee231ae88759462284eb29a20172e8df', 'request_id': 'req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.954266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] The token cache is being invalidate because project ee231ae88759462284eb29a20172e8df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:34.954621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.955711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:34.960241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 932/1884] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:34 2022] DELETE /identity/v3/projects/ee231ae88759462284eb29a20172e8df => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:34.960241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.960241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:34.970018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 953/1885] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:34.970999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:34.984845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5262a-c227-420b-a7aa-47463eccebad None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.011583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:35.087234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fabb13-31ba-48d3-a6b2-2e405f137ecb admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.094772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.095531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.095944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.096416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.097213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.099061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.114185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 933/1886] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:35.115473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.118000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.118354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.118634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.118907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] PATH_INFO: `/v3/users/38ee9b445c6f4a9aa96568883e643272` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.121089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:35.121523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38ee9b445c6f4a9aa96568883e643272)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.123493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.147974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717c0de-5de9-4412-8278-ccfde1368285 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.158539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.159219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.159501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.159774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.160672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.162837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.180346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:35.223198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 934/1887] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:35 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:35.223198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.245596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38ee9b445c6f4a9aa96568883e643272'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:35.247050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38ee9b445c6f4a9aa96568883e643272'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:35.248853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38ee9b445c6f4a9aa96568883e643272'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:35.265010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38ee9b445c6f4a9aa96568883e643272'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:35.277026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38ee9b445c6f4a9aa96568883e643272'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:35.305395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 954/1888] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:34 2022] DELETE /identity/v3/users/38ee9b445c6f4a9aa96568883e643272 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:35.306453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.317928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:35.384006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4ec69-7262-4439-9a83-c8ee44e15a3b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.439320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51e00b-10ae-44a5-99c4-5665a69a0ef7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.445497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.446182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.446532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.446862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.447585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.449845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.480343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.481230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.481909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.482272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.483404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 955/1889] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:35 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:35.483489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.487045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.487837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.504610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40035e-6ef0-44c0-b523-3916feda3957 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.510558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.510758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 935/1890] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:35.511309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.512159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.512159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.512870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.515969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.516319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.518730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:35.534769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 956/1891] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:35.536677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.547329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94644c46-e5ba-4d01-83df-e23e7de2a787 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.550637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5da5-b5a5-4575-842d-6c410c3159d8 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.553046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.554034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.554655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.554922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] PATH_INFO: `/v3/projects/5d8ef1b613ab423f82d6acda84c03c2e/users/5653b834d24e4e5bbb2f6e2098ddf7b9/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.556066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.557668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.557952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.558398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] PATH_INFO: `/v3/projects/98cda1b213244559a15b9f63087ed5f8/users/6fe2301ee0cf41e397bced7fa11da6e6/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.576146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cda1b213244559a15b9f63087ed5f8, user_id=6fe2301ee0cf41e397bced7fa11da6e6, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.590317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.603198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8ef1b613ab423f82d6acda84c03c2e, user_id=5653b834d24e4e5bbb2f6e2098ddf7b9, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.612576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:35.612819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d3abc-1b80-4a97-93b1-aef6d735e9fe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.622974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 957/1892] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:35 2022] PUT /identity/v3/projects/98cda1b213244559a15b9f63087ed5f8/users/6fe2301ee0cf41e397bced7fa11da6e6/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:35.626136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.662983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 936/1893] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:35 2022] PUT /identity/v3/projects/5d8ef1b613ab423f82d6acda84c03c2e/users/5653b834d24e4e5bbb2f6e2098ddf7b9/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:35.675142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.676209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.676552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.767528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] MFA Rules not processed for user `5653b834d24e4e5bbb2f6e2098ddf7b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:35.774621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:35.805928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80274f70-1eff-4f62-9888-963f116030c5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.933398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:35.949914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 937/1894] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:35.950966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.954986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:35.955852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:35.956008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:35.956270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:35.957315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:35.962870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:35.978227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 958/1895] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:35 2022] GET /identity/v3/roles => generated 1420 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:35.979796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11181083-ff0c-40bc-b69f-9a967ff15f67 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:35.980555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:35.996749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386cfa9-dd05-47f9-869e-74324bf7499f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.003857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.004776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.004887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.005305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.005677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.006678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.006966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.007249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] PATH_INFO: `/v3/projects/98cda1b213244559a15b9f63087ed5f8/users/6fe2301ee0cf41e397bced7fa11da6e6/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.009493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98cda1b213244559a15b9f63087ed5f8, user_id=6fe2301ee0cf41e397bced7fa11da6e6, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.012891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.014197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.015040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f425ab9-d48c-44e3-ab33-74d33311bdae admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.027655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 959/1896] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:35 2022] PUT /identity/v3/projects/98cda1b213244559a15b9f63087ed5f8/users/6fe2301ee0cf41e397bced7fa11da6e6/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:36.028540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.035029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.047282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 938/1897] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:35 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:36.048709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.049055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.049988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.100110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] MFA Rules not processed for user `6fe2301ee0cf41e397bced7fa11da6e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:36.101600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.158723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe56ce9-692f-4d8e-ad66-3f84b6423de8 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.194777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 939/1898] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:36.195823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.203737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:36.241669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.242984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.243365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.244242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] PATH_INFO: `/v3/projects/bcf646f22cc6410aac94eb976f3fe05e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.245499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf646f22cc6410aac94eb976f3fe05e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.247466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.275411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e4f80e-5d96-43c7-9340-2220b66264aa None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.281586 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.282262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.282546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.282830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.283510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.285548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.295461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf646f22cc6410aac94eb976f3fe05e', 'request_id': 'req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:36.301409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf646f22cc6410aac94eb976f3fe05e', 'request_id': 'req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:36.307391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 940/1899] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:36.308699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.309977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf646f22cc6410aac94eb976f3fe05e', 'request_id': 'req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:36.311117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.321296 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] The token cache is being invalidate because project bcf646f22cc6410aac94eb976f3fe05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:36.321695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:36.322333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:36.324408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 960/1900] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:36 2022] DELETE /identity/v3/projects/bcf646f22cc6410aac94eb976f3fe05e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:36.325162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.326421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.373158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d94b7-b448-4720-9666-3915d50f9d0b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.376854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:36.391963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7cbe0-2e73-4d97-907e-0f844380ebe0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.398529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.399199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.399479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.399751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.400508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.403488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138e083-6427-4455-bb78-8b9cf86941da admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.413599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 961/1901] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:36.414771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.415053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.415324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.426068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.426723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.427001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.427402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.429568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.434179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:36.435349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.443588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 962/1902] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:36.451647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.459879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278de782-8f55-4e82-8c8d-517f2ea8294e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.465630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.467017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.467017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.467017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.467541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.469162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73526622-1531-4dac-b82b-e58a40048f16 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.480466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 963/1903] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:36.481621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.481911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.482189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.497894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.498708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.502891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:36.505136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.513299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 941/1904] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:36.514263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.516216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 964/1905] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:36.517157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.529621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2e3e29-c43e-4f44-8797-37957381829a None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.534823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3e1f7-e30b-4559-ace2-c7e761342048 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.542367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.543192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.544179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.544665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.544850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] PATH_INFO: `/v3/projects/8f14de31d23d4aef91d8df336bb2949c/users/579756abe1144b46b5e88345ab134bd5/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.545210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.546355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.548262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.549259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:36.550696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.553501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.564933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f14de31d23d4aef91d8df336bb2949c, user_id=579756abe1144b46b5e88345ab134bd5, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.568489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cacfe-3cec-472f-93eb-23203889ed51 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.575876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 965/1906] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:36.581059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.591874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 942/1907] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:36 2022] PUT /identity/v3/projects/8f14de31d23d4aef91d8df336bb2949c/users/579756abe1144b46b5e88345ab134bd5/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:36.597001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522cd798-ba51-4be1-a77f-f213b7ea1784 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.605137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.605423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.605883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.656090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] MFA Rules not processed for user `579756abe1144b46b5e88345ab134bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:36.657471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.702095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:36.777052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 943/1908] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:36.778909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.781260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.793291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.794142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.794493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.794837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.796061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.798339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.814163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:36.846612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 966/1909] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/projects => generated 379 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:36.847855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:36.868289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a3e97-1540-458f-bf25-49a7bacd9ce6 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.935298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2480aa8-b9e0-4fce-b231-8c42d799396a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:36.940158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.940860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.941140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.941415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.942113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:36.944009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:36.950941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:36.952374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:36.952971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:36.952971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:36.956463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:36.976019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 967/1910] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/projects => generated 387 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:36.977034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.052361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.056334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.071935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 944/1911] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:37.073492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.088360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0da592-ef16-4735-9126-e4f9a87c9924 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.089667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82a73-283f-4024-a616-1c2fc32257e6 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.093629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.094449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.094677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.094915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.095606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.098476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.104610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.107040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.107419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.107800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.109000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.115317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.135703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 968/1912] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:36 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:37.140575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.152717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249da034-3009-4b5b-8588-a63d6b600074 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.157330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 945/1913] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:37 2022] POST /identity/v3/users => generated 300 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:37.159442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.160105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.160480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.160769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.162367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.162859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.164564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.175770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbf101-87eb-4e2a-926c-3e5016815a6e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.177942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 969/1914] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:37.181804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.189816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.190504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.190785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.191058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.191784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.193425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.203962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7ed81-7abd-4db0-8284-2fec8f1ac7e2 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.208388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 946/1915] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:37.212561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.216090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.217049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.218362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.218883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.221438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.224896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995ed1f-a859-4c4b-b1cb-c22872cf2cfe admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.226474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.230529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.231583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.231863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.232138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] PATH_INFO: `/v3/projects/9c68d40a41324e1d800ca35d304d2c4a/users/601a06ce08b34323a78a9862b275b123/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.251264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68d40a41324e1d800ca35d304d2c4a, user_id=601a06ce08b34323a78a9862b275b123, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.253959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.272104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 947/1916] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:37 2022] PUT /identity/v3/projects/9c68d40a41324e1d800ca35d304d2c4a/users/601a06ce08b34323a78a9862b275b123/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:37.278076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.279971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 970/1917] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:37.285540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.293499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:37.309515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ddd59a-a202-46a4-83c3-a1b0ddd55b85 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.332892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:37.384106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eac5c-578f-45aa-8c43-702462c8d4b3 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.401849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.403490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.403759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.404040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.405746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.407610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.408044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.408646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] PATH_INFO: `/v3/projects/ac7f8034a7814303ad7cf6d2c8800d4a/users/6b281f430e12448592e5d9e1b963f6dd/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.411187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.413299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.431818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 948/1918] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:37.433023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac7f8034a7814303ad7cf6d2c8800d4a, user_id=6b281f430e12448592e5d9e1b963f6dd, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.437092 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec85d51-a618-4555-ac40-6a57fc2ee9f9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.438353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.458751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753cb9a-a9db-4207-b98d-c301c4001953 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.464635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 971/1919] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:37 2022] PUT /identity/v3/projects/ac7f8034a7814303ad7cf6d2c8800d4a/users/6b281f430e12448592e5d9e1b963f6dd/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:37.473334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.473773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.474779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.551347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] MFA Rules not processed for user `6b281f430e12448592e5d9e1b963f6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:37.554215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:37.589842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.591606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.591968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.593108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] PATH_INFO: `/v3/projects/9c68d40a41324e1d800ca35d304d2c4a/users/601a06ce08b34323a78a9862b275b123/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.597550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c68d40a41324e1d800ca35d304d2c4a, user_id=601a06ce08b34323a78a9862b275b123, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.604841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba2ee2-2336-482f-8a77-46deb7ac30a0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.638961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 949/1920] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:37 2022] PUT /identity/v3/projects/9c68d40a41324e1d800ca35d304d2c4a/users/601a06ce08b34323a78a9862b275b123/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:37.650968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2c63f-7599-46d8-827d-441482351248 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.651581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2c63f-7599-46d8-827d-441482351248 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.652213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2c63f-7599-46d8-827d-441482351248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.696882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-29c2c63f-7599-46d8-827d-441482351248 None None] MFA Rules not processed for user `601a06ce08b34323a78a9862b275b123`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:37.698901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 972/1921] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:37.702672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-29c2c63f-7599-46d8-827d-441482351248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:37.703334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.718665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ad78b-e7a5-466f-8857-77d851ed904c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.791748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.792545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.792829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.793157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.793911 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.795843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.800716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 950/1922] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:37.812857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2c63f-7599-46d8-827d-441482351248 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.814402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-29c2c63f-7599-46d8-827d-441482351248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:37.818424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:37.828377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 973/1923] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:37 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:37.832236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.883426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2c63f-7599-46d8-827d-441482351248 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.894066 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:37.909864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaabb15-2236-4dad-81d9-073738d48f41 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.915609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.915944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.917029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.917383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.918649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.920704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.945156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 974/1924] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:37 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:37.948048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.948754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.948960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.949063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.949295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.969435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae063c81-cdc5-44f9-93d5-7acec0b4a271 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:37.969435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:37.969435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:37.969435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:37.969818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:37.969818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:37.969818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:37.981201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 975/1925] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:37.985287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:37.997049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cca804-8590-4416-8320-d05093f0c366 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.003284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.003980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.004267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.004591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.005341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.007161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.022702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 976/1926] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:38.024241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.025190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.027334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b81e1-4c72-42a7-b12e-10f865fa0321 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.038712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 951/1927] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:38.039752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942782e-8ebc-4fd4-ad5a-cf3f437dd5c4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.045239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.046162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.046438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.046715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] PATH_INFO: `/v3/projects/e2bd8c2a2b7046238a85258e71ebfe98/users/27904ebcd7ea41fd950283833cb9606f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.063498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bd8c2a2b7046238a85258e71ebfe98, user_id=27904ebcd7ea41fd950283833cb9606f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.066219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.088935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 977/1928] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:38 2022] PUT /identity/v3/projects/e2bd8c2a2b7046238a85258e71ebfe98/users/27904ebcd7ea41fd950283833cb9606f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:38.094022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.094851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.094851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.128905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] MFA Rules not processed for user `27904ebcd7ea41fd950283833cb9606f`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:38.130382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:38.210484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 952/1929] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:38.215050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.284889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc67c65-59fb-4419-8547-2515d595dc9b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.301629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.302516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.302958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.303361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.304199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.306466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.355788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 978/1930] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:38 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:38.357489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.369316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd38c1-1ff9-4ffd-a33a-555ede5681a0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.424448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:38.446551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.447419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.454772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.455074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.460336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.472455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.527147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 953/1931] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:38 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:38.528913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.550499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e1761-1780-4321-9bd5-f40c9b01a3d5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.579935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.579935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.579935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.579935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.585869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.587845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.628240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 979/1932] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:38.629588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.645350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518d23a-8e5e-4127-923d-53367659d8ce admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.652999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.654058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.654383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.654671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] PATH_INFO: `/v3/projects/7d7abf4cd90543f689a15b968e9b6590/users/1732e167f7004a178d26d863efb34e3f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.687413 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7abf4cd90543f689a15b968e9b6590, user_id=1732e167f7004a178d26d863efb34e3f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.690487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.716462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 954/1933] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:38 2022] PUT /identity/v3/projects/7d7abf4cd90543f689a15b968e9b6590/users/1732e167f7004a178d26d863efb34e3f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:38.721303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.736943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388531a9-885c-4da8-b90f-d730ff4a8693 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.778377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:38.858576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.859337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.859583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.859854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.861027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.862709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36528848-7e9c-46e3-bca5-2f3b611c4663 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.882087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 980/1934] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:38.883556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.900899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86678c-c681-4ccf-9453-e8f919fbe0e6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:38.911029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:38.912721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.913394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.913912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] PATH_INFO: `/v3/projects/7d7abf4cd90543f689a15b968e9b6590/users/1732e167f7004a178d26d863efb34e3f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.918571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7abf4cd90543f689a15b968e9b6590, user_id=1732e167f7004a178d26d863efb34e3f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:38.924481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:38.940270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 955/1935] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:38 2022] PUT /identity/v3/projects/7d7abf4cd90543f689a15b968e9b6590/users/1732e167f7004a178d26d863efb34e3f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:38.951904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:38.952461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:38.952839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:38.990810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:38.993746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] MFA Rules not processed for user `1732e167f7004a178d26d863efb34e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:38.995365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:39.122455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 981/1936] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:39.131597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526be9e7-2b95-4bd1-bc04-048cfdbe80fa admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:39.137899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:39.138778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:39.139063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:39.139335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:39.141221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:39.242847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:39.244540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:39.245311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:39.254948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5608a89-4567-4070-a714-d4703cf35ed9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:39.257252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 956/1937] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:39.260917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:39.263183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:39.263596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:39.263883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:39.277938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:39.279625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:39.305792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 982/1938] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:39.539664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:39.552363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54d2ee-8a19-4113-92d0-9201312954fc service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:39.556640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:39.557404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:39.557692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:39.557968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:39.633107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:39.636909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:39.638872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88786fd0-a0bb-41f7-b550-7bf7e81d625f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:39.656898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 957/1939] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:40.406830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:40.408315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:40.475758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d6152-6f77-4ed7-8f03-81e748a8a719 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:40.485648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:40.486527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:40.486809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:40.487082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] PATH_INFO: `/v3/users/9d58372887b14787a831b7d3aa26119c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:40.488264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d58372887b14787a831b7d3aa26119c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:40.490202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:40.522883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:40.545901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d58372887b14787a831b7d3aa26119c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:40.546582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d58372887b14787a831b7d3aa26119c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:40.547323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d58372887b14787a831b7d3aa26119c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:40.560570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d58372887b14787a831b7d3aa26119c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:40.569039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d58372887b14787a831b7d3aa26119c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:40.605610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 983/1940] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:40 2022] DELETE /identity/v3/users/9d58372887b14787a831b7d3aa26119c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:40.610550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:40.610993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:40.611094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:40.638200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:40.639436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:40.756351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 958/1941] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:40.769837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:40.771547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:40.893831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67253c71-97cd-4f92-b1e3-ac18d2835113 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:40.899361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:40.899720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:40.901955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:40.902717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:40.920264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:40.921208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:40.939526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 984/1942] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:41.604485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:41.606081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:41.619221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab06e7-97fd-4956-bf0a-651fd18f957b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:41.626419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:41.627573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:41.628008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:41.628274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] PATH_INFO: `/v3/projects/8e90ebb6ca4441a0a0c6df5dcff8eb4e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:41.630099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e90ebb6ca4441a0a0c6df5dcff8eb4e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:41.632731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:41.684421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e90ebb6ca4441a0a0c6df5dcff8eb4e', 'request_id': 'req-f8b0e177-8b50-432f-9570-7fe080926c50'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.690593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e90ebb6ca4441a0a0c6df5dcff8eb4e', 'request_id': 'req-f8b0e177-8b50-432f-9570-7fe080926c50'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.698767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e90ebb6ca4441a0a0c6df5dcff8eb4e', 'request_id': 'req-f8b0e177-8b50-432f-9570-7fe080926c50'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.713775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] The token cache is being invalidate because project 8e90ebb6ca4441a0a0c6df5dcff8eb4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:41.714138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.715113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.716922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 959/1943] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:41 2022] DELETE /identity/v3/projects/8e90ebb6ca4441a0a0c6df5dcff8eb4e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:41.721055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:41.732055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa76725-8d3d-4f2e-86ec-a229402677ee service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:41.788493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:41.789369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:41.789646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:41.789919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] PATH_INFO: `/v3/users/da17193e2feb4f2ca5f144ad5d706b36` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:41.791087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=da17193e2feb4f2ca5f144ad5d706b36)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:41.793610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:41.855489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da17193e2feb4f2ca5f144ad5d706b36'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.856180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da17193e2feb4f2ca5f144ad5d706b36'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.856835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da17193e2feb4f2ca5f144ad5d706b36'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.859950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:41.867225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da17193e2feb4f2ca5f144ad5d706b36'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.875475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-13c6df4b-8acb-4e45-b4a4-a8bdf09134ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da17193e2feb4f2ca5f144ad5d706b36'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:41.894368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 985/1944] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:41 2022] DELETE /identity/v3/users/da17193e2feb4f2ca5f144ad5d706b36 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:42.193716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.197198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:42.225574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0e177-8b50-432f-9570-7fe080926c50 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.324904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.326132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.326530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.327029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] PATH_INFO: `/v3/projects/d6babfc1283445d98e8b04f6521bce3e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.328524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6babfc1283445d98e8b04f6521bce3e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.330790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.373130 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:42.420822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6babfc1283445d98e8b04f6521bce3e', 'request_id': 'req-51cf2007-03d9-4c50-a333-47dffb0d4d17'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.427359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6babfc1283445d98e8b04f6521bce3e', 'request_id': 'req-51cf2007-03d9-4c50-a333-47dffb0d4d17'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.435783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6babfc1283445d98e8b04f6521bce3e', 'request_id': 'req-51cf2007-03d9-4c50-a333-47dffb0d4d17'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.447213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] The token cache is being invalidate because project d6babfc1283445d98e8b04f6521bce3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:42.448241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.448896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51cf2007-03d9-4c50-a333-47dffb0d4d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.450623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 960/1945] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:42 2022] DELETE /identity/v3/projects/d6babfc1283445d98e8b04f6521bce3e => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:42.455444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.455903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.456183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.475209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:42.476438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:42.555269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 986/1946] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:42.563138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.563509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.563935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.597082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:42.608163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:42.619780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 961/1947] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:42.625044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.643107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035a543-d396-41c4-985c-67df128a1aa2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.648007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.648717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.648997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.649270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.650005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:42.650310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.652103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.668346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 987/1948] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:42.670552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.679796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1403175-2757-4cff-918a-ce4653d9b7bf None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.691077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.692136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.692320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.692897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.694104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.696691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.738018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.739712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:42.743600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 962/1949] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:42 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:42.747668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.753546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea20c84-6cc2-44c2-b6b5-1db18a4f6c3f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.825058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ba7f7-93af-4620-896a-dba00c6873fc admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.833411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.834097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.834571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.834723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.835374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.837433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.843140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.844763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.845050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.845574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] PATH_INFO: `/v3/users/d0a7dd85292b40d7817c8e0ce9d956ca` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.847667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0a7dd85292b40d7817c8e0ce9d956ca)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.850529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.872320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 963/1950] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:42 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:42.876148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.888365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4e9ae-8b73-47d6-88b2-749750c00e17 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.894604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.895367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.895722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.896059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.896890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.898599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.908936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0a7dd85292b40d7817c8e0ce9d956ca'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.909626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0a7dd85292b40d7817c8e0ce9d956ca'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.910224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0a7dd85292b40d7817c8e0ce9d956ca'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.911237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 964/1951] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:42.915372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.918067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:42.919932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0a7dd85292b40d7817c8e0ce9d956ca'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.922837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:42.930401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0a7dd85292b40d7817c8e0ce9d956ca'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:42.935347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd53866-65b7-4e15-a6d9-c1c54788ff0d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:42.940463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:42.941510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:42.941859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:42.942347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] PATH_INFO: `/v3/users/6b281f430e12448592e5d9e1b963f6dd` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:42.943803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b281f430e12448592e5d9e1b963f6dd)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:42.945841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:42.953150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 988/1952] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:42 2022] DELETE /identity/v3/users/d0a7dd85292b40d7817c8e0ce9d956ca => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:42.954529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:42.956010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.016498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b281f430e12448592e5d9e1b963f6dd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:43.017207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b281f430e12448592e5d9e1b963f6dd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:43.018507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b281f430e12448592e5d9e1b963f6dd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:43.025533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=14,keystone:SELECT=81,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:43.028009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b281f430e12448592e5d9e1b963f6dd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:43.040802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4e6c7-a3fc-450c-8804-64babba5b69b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.049018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.049222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-63123e49-71ac-4d1b-a9b2-3f5db29b23da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b281f430e12448592e5d9e1b963f6dd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:43.049907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.050201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.050480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.051504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.053934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77187c42-cd06-4323-b71f-5a10ead4061a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.066295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 989/1954] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:43.067927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.068225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.068522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.070383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 965/1954] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:42 2022] DELETE /identity/v3/users/6b281f430e12448592e5d9e1b963f6dd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:43.074315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.074610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.074893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.091154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:43.092593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.102610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:43.104582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.175691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 990/1955] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:43.175691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.175691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.181811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113bf04-44ab-4c41-b136-ef4a352ecf7b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.186690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.187583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.187867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.188143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] PATH_INFO: `/v3/projects/061f1577a1064ba88cdb94665b724762/users/14d255f9467140378ba82b051abb7e54/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.192723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 966/1956] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:43.195861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.195861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.205128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=061f1577a1064ba88cdb94665b724762, user_id=14d255f9467140378ba82b051abb7e54, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.207791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.222022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 991/1957] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:43 2022] PUT /identity/v3/projects/061f1577a1064ba88cdb94665b724762/users/14d255f9467140378ba82b051abb7e54/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:43.222809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.224416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.266523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2ea8a-52f8-4b1e-9704-455f2cb64574 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.289645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:43.296807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16209cb3-69f1-45f9-9516-0ad51f361f7b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.308111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.308855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.309756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.309756 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.332817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.333751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.334030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.334299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.382045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.383999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef21ee-b523-4e3d-9693-331a000fd48c service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.386841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:43.398064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 992/1958] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:43.399681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1502-02ee-4aff-a011-301aa257934d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.400012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1502-02ee-4aff-a011-301aa257934d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.401027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c1502-02ee-4aff-a011-301aa257934d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.414934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.417384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.436576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 967/1959] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:43.473201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-341c1502-02ee-4aff-a011-301aa257934d None None] MFA Rules not processed for user `14d255f9467140378ba82b051abb7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:43.474640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-341c1502-02ee-4aff-a011-301aa257934d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.569564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 993/1960] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:43.575279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.586410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d25f7d9-b70c-4bdf-82dd-309f065248b5 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.592487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.593516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.593516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.593912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.594749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.596760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.635568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 968/1961] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:43 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:43.636795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1502-02ee-4aff-a011-301aa257934d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.656562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c1502-02ee-4aff-a011-301aa257934d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.767021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.767726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.768010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.768368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.769001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.773562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.810651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 994/1962] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:43 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:43.814890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.834918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209bd6f-75f0-4ad3-ac8e-d08ca0faf298 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.841865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.842655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.843007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.843874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.844893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.847426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.867078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 969/1963] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:43.868472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.884796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4749dc-56ca-4864-ad88-221e14d54178 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:43.893575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:43.894614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:43.894966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:43.895310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] PATH_INFO: `/v3/projects/987a6c449ac84a8dbf1279e0169d0925/users/2a1833be5d8a47d9a4067da94794660f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:43.910664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:43.916063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=987a6c449ac84a8dbf1279e0169d0925, user_id=2a1833be5d8a47d9a4067da94794660f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:43.919930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:43.949474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 995/1964] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:43 2022] PUT /identity/v3/projects/987a6c449ac84a8dbf1279e0169d0925/users/2a1833be5d8a47d9a4067da94794660f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:43.949725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.949725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:43.954220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:43.973116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e624e5-04f9-4525-b15c-de0eba0aaf89 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.037534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26936bfd-af39-49df-8edc-cc154edff9eb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.045994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.046681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.046968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.047242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.047977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.050074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.060465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.061339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.061630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.061898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] PATH_INFO: `/v3/projects/7694a9dfbef5453db04f7973f32f3455` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.066101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 996/1965] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:44.066259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7694a9dfbef5453db04f7973f32f3455)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.066348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.066441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.069033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.132714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7694a9dfbef5453db04f7973f32f3455', 'request_id': 'req-c49a974e-6ab6-4d55-ad49-2057e86a814a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.139936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7694a9dfbef5453db04f7973f32f3455', 'request_id': 'req-c49a974e-6ab6-4d55-ad49-2057e86a814a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.144002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea5e5d-e46c-446a-8829-4a3e7ebefafe admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.149096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7694a9dfbef5453db04f7973f32f3455', 'request_id': 'req-c49a974e-6ab6-4d55-ad49-2057e86a814a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.162803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] The token cache is being invalidate because project 7694a9dfbef5453db04f7973f32f3455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:44.163250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.164347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.168444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 970/1966] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:43 2022] DELETE /identity/v3/projects/7694a9dfbef5453db04f7973f32f3455 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:44.168444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.169755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.203147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.203883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.204171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.204619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.206522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.250655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:44.285115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49a974e-6ab6-4d55-ad49-2057e86a814a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.289339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.290978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.302667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.303683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.304032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.304328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] PATH_INFO: `/v3/projects/987a6c449ac84a8dbf1279e0169d0925/users/2a1833be5d8a47d9a4067da94794660f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.307961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987a6c449ac84a8dbf1279e0169d0925, user_id=2a1833be5d8a47d9a4067da94794660f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.309031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:44.311592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.322570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 997/1967] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:44.326325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 971/1968] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:44 2022] PUT /identity/v3/projects/987a6c449ac84a8dbf1279e0169d0925/users/2a1833be5d8a47d9a4067da94794660f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:44.339605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.341024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.345954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.347034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.386522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede0fd5-bd71-48b8-8524-834959d35008 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.463956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.464903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.465180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.465449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] PATH_INFO: `/v3/users/6fe2301ee0cf41e397bced7fa11da6e6` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.466707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe2301ee0cf41e397bced7fa11da6e6)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.469969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.485317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b18c02-3415-4eef-bc4b-7a71216288f2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.494977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.496978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.497268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.497699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] PATH_INFO: `/v3/projects/ac7f8034a7814303ad7cf6d2c8800d4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.499612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac7f8034a7814303ad7cf6d2c8800d4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.503000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.534421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe2301ee0cf41e397bced7fa11da6e6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.535507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe2301ee0cf41e397bced7fa11da6e6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.536572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe2301ee0cf41e397bced7fa11da6e6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.548051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe2301ee0cf41e397bced7fa11da6e6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.558507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4a18d9af-a24d-4fb1-8906-dd5313315948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe2301ee0cf41e397bced7fa11da6e6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.571575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac7f8034a7814303ad7cf6d2c8800d4a', 'request_id': 'req-35796d55-7506-48f4-a00c-6039c9329c2b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.579037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac7f8034a7814303ad7cf6d2c8800d4a', 'request_id': 'req-35796d55-7506-48f4-a00c-6039c9329c2b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.582427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 998/1969] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:44 2022] DELETE /identity/v3/users/6fe2301ee0cf41e397bced7fa11da6e6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:44.583989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.584514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.585105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.594068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac7f8034a7814303ad7cf6d2c8800d4a', 'request_id': 'req-35796d55-7506-48f4-a00c-6039c9329c2b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.613565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] The token cache is being invalidate because project ac7f8034a7814303ad7cf6d2c8800d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:44.613867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.614640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.622487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 972/1970] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:44 2022] DELETE /identity/v3/projects/ac7f8034a7814303ad7cf6d2c8800d4a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:44.625677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.627244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.646283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] MFA Rules not processed for user `2a1833be5d8a47d9a4067da94794660f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:44.649315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.716590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35796d55-7506-48f4-a00c-6039c9329c2b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.721641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:44.831240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.832275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.832636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.832853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] PATH_INFO: `/v3/users/27904ebcd7ea41fd950283833cb9606f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.836924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27904ebcd7ea41fd950283833cb9606f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:44.841401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:44.846416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 999/1971] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:44.848976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:44.851974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:44.864791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:44.980433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aabf4e-2fa2-4b58-89a4-1e6ac9a719ac None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:44.986938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:44.987667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:44.988034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27904ebcd7ea41fd950283833cb9606f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.988147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:44.988690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:44.989145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27904ebcd7ea41fd950283833cb9606f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.990864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27904ebcd7ea41fd950283833cb9606f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:44.992632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.012826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27904ebcd7ea41fd950283833cb9606f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.033709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:45.035509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:45.036838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27904ebcd7ea41fd950283833cb9606f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.053701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1000/1972] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:45.059389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 973/1973] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:44 2022] DELETE /identity/v3/users/27904ebcd7ea41fd950283833cb9606f => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:45.210907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:45.213681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.231416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa3b97-883b-4634-a400-493bd24a6f4e service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:45.329024 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:45.330458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:45.330797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:45.331081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:45.334083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.416904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:45.418580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:45.443815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1001/1974] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:45.474762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:45.476661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.652976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867ddc9-ce41-4aff-9c52-9b7dd12dc852 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:45.664768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:45.665775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:45.666042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:45.666320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] PATH_INFO: `/v3/projects/e2bd8c2a2b7046238a85258e71ebfe98` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:45.676479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bd8c2a2b7046238a85258e71ebfe98)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:45.679710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:45.690060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:45.692063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.741140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93056670-8633-4e98-ae80-588fba8a7765 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:45.750832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:45.751915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:45.752423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:45.752777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] PATH_INFO: `/v3/projects/98cda1b213244559a15b9f63087ed5f8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:45.754626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98cda1b213244559a15b9f63087ed5f8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:45.757648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:45.773888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:45.808188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bd8c2a2b7046238a85258e71ebfe98', 'request_id': 'req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.816101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bd8c2a2b7046238a85258e71ebfe98', 'request_id': 'req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.828328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bd8c2a2b7046238a85258e71ebfe98', 'request_id': 'req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.845735 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98cda1b213244559a15b9f63087ed5f8', 'request_id': 'req-9fbb63a1-f4f6-4808-9619-4974314e28d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.847745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] The token cache is being invalidate because project e2bd8c2a2b7046238a85258e71ebfe98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:45.848263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.849120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.850769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:45.852813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98cda1b213244559a15b9f63087ed5f8', 'request_id': 'req-9fbb63a1-f4f6-4808-9619-4974314e28d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.863587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 974/1975] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:45 2022] DELETE /identity/v3/projects/e2bd8c2a2b7046238a85258e71ebfe98 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:45.867199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:45.868703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98cda1b213244559a15b9f63087ed5f8', 'request_id': 'req-9fbb63a1-f4f6-4808-9619-4974314e28d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.869312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.888937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] The token cache is being invalidate because project 98cda1b213244559a15b9f63087ed5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:45.889361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.890434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9fbb63a1-f4f6-4808-9619-4974314e28d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:45.892167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1002/1976] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:45 2022] DELETE /identity/v3/projects/98cda1b213244559a15b9f63087ed5f8 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:45.897750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559a0f-ebf3-4680-a776-8b64576b6e37 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:45.898153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559a0f-ebf3-4680-a776-8b64576b6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:45.898529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62559a0f-ebf3-4680-a776-8b64576b6e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:45.927794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-62559a0f-ebf3-4680-a776-8b64576b6e37 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:45.929522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-62559a0f-ebf3-4680-a776-8b64576b6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:45.955051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c74db-e8bd-41cd-83f8-740adcf17e5e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.019093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1003/1977] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:46.027800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.028368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.028743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.033438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.034384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.034665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.034940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] PATH_INFO: `/v3/users/1732e167f7004a178d26d863efb34e3f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.036735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1732e167f7004a178d26d863efb34e3f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.039355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.054531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:46.055760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.068272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1004/1978] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:46.069310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.085944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b244ed-ec46-4e6e-bee7-9a123a6f09f1 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.092349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.093153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.093775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.094135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.094979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:46.095380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.097337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.112897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1732e167f7004a178d26d863efb34e3f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.113591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1732e167f7004a178d26d863efb34e3f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.114213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1732e167f7004a178d26d863efb34e3f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.118202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1005/1979] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:46.120062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.122693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1732e167f7004a178d26d863efb34e3f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.123007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.132449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:46.139791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1732e167f7004a178d26d863efb34e3f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.139997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e9825-d4b3-45d5-93f8-f0e3f608b4ce admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.145524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.146364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.146650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.147112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.147890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.149892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.157375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 975/1980] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:45 2022] DELETE /identity/v3/users/1732e167f7004a178d26d863efb34e3f => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:20:46.181316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1006/1981] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:46.186095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.268191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816b72a-99ab-43c6-871b-a57c91e173d3 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.345499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.346182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.346461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.346731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.347483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.349811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.379386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 976/1982] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:46.383312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.398100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227c75c-ca02-416e-9b1e-e4a595fd9720 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.404208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926df658-2377-4631-a232-6320dc368643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.405349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926df658-2377-4631-a232-6320dc368643 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.405709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926df658-2377-4631-a232-6320dc368643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.406057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926df658-2377-4631-a232-6320dc368643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.406865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926df658-2377-4631-a232-6320dc368643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.408970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926df658-2377-4631-a232-6320dc368643 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.423623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1007/1983] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:46.424939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.458011 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3bd9f1-9579-47b6-bd92-06a72f720251 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.464626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.465470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.465806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.466135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.468172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926df658-2377-4631-a232-6320dc368643 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.470744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-926df658-2377-4631-a232-6320dc368643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.474966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.483487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.499333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926df658-2377-4631-a232-6320dc368643 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.502842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 977/1984] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:46.506699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.515944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.516913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.517254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.517458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] PATH_INFO: `/v3/projects/7d7abf4cd90543f689a15b968e9b6590` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.518980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7abf4cd90543f689a15b968e9b6590)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.520989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.521403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09290b-88ef-4cfc-b40d-4fdc912636cf admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.531840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.531840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.531840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.531840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] PATH_INFO: `/v3/projects/9b7511ddb53845b5a412211281623cc1/users/2e5044706bb84effad2beb4d0d57480e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.548798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7511ddb53845b5a412211281623cc1, user_id=2e5044706bb84effad2beb4d0d57480e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.551465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48874339-157f-4f81-ad6b-fe9e9ad2e721 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.570889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 978/1985] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:46 2022] PUT /identity/v3/projects/9b7511ddb53845b5a412211281623cc1/users/2e5044706bb84effad2beb4d0d57480e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:46.582457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.582739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.583007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.598122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7abf4cd90543f689a15b968e9b6590', 'request_id': 'req-28e124c9-9f70-4926-8a89-79378f848918'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.604823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7abf4cd90543f689a15b968e9b6590', 'request_id': 'req-28e124c9-9f70-4926-8a89-79378f848918'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.612547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] MFA Rules not processed for user `2e5044706bb84effad2beb4d0d57480e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:46.613982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.625343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7abf4cd90543f689a15b968e9b6590', 'request_id': 'req-28e124c9-9f70-4926-8a89-79378f848918'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.646119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] The token cache is being invalidate because project 7d7abf4cd90543f689a15b968e9b6590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:46.646426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.659075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-28e124c9-9f70-4926-8a89-79378f848918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:46.659075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1008/1986] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:46 2022] DELETE /identity/v3/projects/7d7abf4cd90543f689a15b968e9b6590 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:46.660784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:46.661971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f824f-8518-4d9c-972a-78b5bf6e30ed None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.662268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f824f-8518-4d9c-972a-78b5bf6e30ed None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.662636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f824f-8518-4d9c-972a-78b5bf6e30ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.686315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-229f824f-8518-4d9c-972a-78b5bf6e30ed None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:46.687572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-229f824f-8518-4d9c-972a-78b5bf6e30ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.696479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:46.709277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 979/1987] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:46.722552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.724842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.804143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1009/1988] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:46.814026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.814318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.814597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.849641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:46.851286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:46.861105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238f93ab-4ad7-45c7-9602-d6bab9b8b0bf None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.867119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.868437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.870400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.870860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.873692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1010/1989] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:46.878228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.888793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617aa6e-5f70-4b2d-8f61-6fd8134fbcf0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.894013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.894694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.894981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.895261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.896003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:46.896328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.897964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.906383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.912182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.914616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1011/1990] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:46.919425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.935408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 980/1991] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:46.938161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06bdc07-b37f-4f24-8c1f-3f1a84370115 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:46.948748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:46.949478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:46.949769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:46.950045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:46.953380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:46.958884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:46.961220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:46.966912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.001404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0bbb-3ef4-4c4c-9dd0-8da52a64788a service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.012962 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.014351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.014502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.014785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] PATH_INFO: `/v3/users/2e5044706bb84effad2beb4d0d57480e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.016514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5044706bb84effad2beb4d0d57480e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.019698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.027463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1012/1992] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:46 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:47.034468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.116687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5044706bb84effad2beb4d0d57480e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.117697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5044706bb84effad2beb4d0d57480e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.118672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5044706bb84effad2beb4d0d57480e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.136337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5044706bb84effad2beb4d0d57480e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.140948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb67835-11a3-4c22-ab88-9e2ad9759711 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.146643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.147434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.147779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.148150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.149166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.151763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.162316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-566c9818-8c19-4703-979d-d466f9137ebb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5044706bb84effad2beb4d0d57480e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.175431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1013/1993] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:47 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:47.180217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.183923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.186389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 981/1994] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:46 2022] DELETE /identity/v3/users/2e5044706bb84effad2beb4d0d57480e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:47.191443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1259-777f-47b5-9877-890229783156 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.196848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1259-777f-47b5-9877-890229783156 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.197171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe1259-777f-47b5-9877-890229783156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.207354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871a113-06ee-49c3-9df2-9200c8a28b82 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.228214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-24fe1259-777f-47b5-9877-890229783156 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:47.230142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-24fe1259-777f-47b5-9877-890229783156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.298386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:47.325840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 982/1995] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:47.343902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1259-777f-47b5-9877-890229783156 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.348181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-24fe1259-777f-47b5-9877-890229783156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.353572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.354526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.357203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.357497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.361247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.364563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.371699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:47.396727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1014/1996] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:47.399917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.411451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9dde-7ec8-4a16-9df0-07d4e55a3b4d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.418467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.419427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.419801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.420174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.421033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.422993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.434912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1015/1997] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:47.437539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe1259-777f-47b5-9877-890229783156 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.439625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.442134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.442802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.443090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.443376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.455079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.456622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d1697-d520-49e5-9273-518e88e08603 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.459820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b244-2f9e-4695-81b5-1510d258c8ca admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.466453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.467898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.468064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 983/1998] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:47.468572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.468897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] PATH_INFO: `/v3/projects/031bd1ae0ca84ac09b2fc030436994f5/users/7cd32b5620144b43afd9b349c7054c3e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.488491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031bd1ae0ca84ac09b2fc030436994f5, user_id=7cd32b5620144b43afd9b349c7054c3e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.491680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.516231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1016/1999] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:47 2022] PUT /identity/v3/projects/031bd1ae0ca84ac09b2fc030436994f5/users/7cd32b5620144b43afd9b349c7054c3e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:47.529943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.530274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.530510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.578284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] MFA Rules not processed for user `7cd32b5620144b43afd9b349c7054c3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:47.579872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.659400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.662030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.680523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 984/2000] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:47.685268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.787452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefe9b4-0bd8-4b9f-8b1d-3e13655e6ecd None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.792837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.794174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.794174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.794463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.795243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.797472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.804087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e110de-bbf4-4838-89f0-53d4324cfe58 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.809974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:47.811198 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:47.811704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:47.812326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] PATH_INFO: `/v3/users/5653b834d24e4e5bbb2f6e2098ddf7b9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:47.813948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5653b834d24e4e5bbb2f6e2098ddf7b9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:47.816694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:47.849572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 985/2001] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:47 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:47.856821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.882404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:47.889848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5653b834d24e4e5bbb2f6e2098ddf7b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.890666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5653b834d24e4e5bbb2f6e2098ddf7b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.891899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5653b834d24e4e5bbb2f6e2098ddf7b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.900995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5653b834d24e4e5bbb2f6e2098ddf7b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.915020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5653b834d24e4e5bbb2f6e2098ddf7b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:47.934490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1017/2002] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:47 2022] DELETE /identity/v3/users/5653b834d24e4e5bbb2f6e2098ddf7b9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:47.935500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:47.936941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:47.944824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d64faa-99bf-4c08-8063-8d849722f556 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:47.983234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.003600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.004283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.004596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.004874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.004996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85811c0-fba4-4cc7-bb81-339d0ec9c0eb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.005737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.008070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a94424-b0b8-4132-8153-124b10ed5d4f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.010646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.012437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.012897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.013176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] PATH_INFO: `/v3/projects/9b7511ddb53845b5a412211281623cc1` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.014421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7511ddb53845b5a412211281623cc1)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.016858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.032860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 986/2003] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:47 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:48.036876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d14cf-c854-41c3-a45b-778b22003664 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.037171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d14cf-c854-41c3-a45b-778b22003664 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.037453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d14cf-c854-41c3-a45b-778b22003664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.057177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2a6d14cf-c854-41c3-a45b-778b22003664 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:48.058386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2a6d14cf-c854-41c3-a45b-778b22003664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.083182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7511ddb53845b5a412211281623cc1', 'request_id': 'req-40a71e91-d7a7-410f-a9c3-2acad5936975'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.091141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7511ddb53845b5a412211281623cc1', 'request_id': 'req-40a71e91-d7a7-410f-a9c3-2acad5936975'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.099045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7511ddb53845b5a412211281623cc1', 'request_id': 'req-40a71e91-d7a7-410f-a9c3-2acad5936975'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.110917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] The token cache is being invalidate because project 9b7511ddb53845b5a412211281623cc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:48.111430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.112442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.116446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1018/2004] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:47 2022] DELETE /identity/v3/projects/9b7511ddb53845b5a412211281623cc1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:48.117542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.119398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.151941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 987/2005] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:48.153443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.153745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.153998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.174850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:48.176093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.185697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 988/2006] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:48.186515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.187774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.208348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a71e91-d7a7-410f-a9c3-2acad5936975 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.220170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.220937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.221284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.221626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.226607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.226607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b529e-b9bd-46bb-ab50-403e9d5a26ef admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.231603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.234906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1019/2007] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:48.239757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.240047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.240343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.247705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5af9c-cece-45b4-a71e-2a2584b0b872 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.251473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.252113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.252427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.252701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.258097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:48.259287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.262385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.263824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.267994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1020/2008] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:48.268655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.276498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 989/2009] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:48.277421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.278428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1eb987-ece1-42ae-a956-9c4f741a6198 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.282926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.283979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.284257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.284669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] PATH_INFO: `/v3/projects/f337839d467943f7a383252f8214b0ec/users/0e6693ca9acb4f1f95338dc10fb728f2/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.287323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701e26b-1f19-4b0d-aa70-7f7d45cd56bb service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.292425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.293146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.293559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.294020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.294762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:48.295051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.296707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.301644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f337839d467943f7a383252f8214b0ec, user_id=0e6693ca9acb4f1f95338dc10fb728f2, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.304285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.312519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 990/2010] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:48.315721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.319950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1021/2011] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:48 2022] PUT /identity/v3/projects/f337839d467943f7a383252f8214b0ec/users/0e6693ca9acb4f1f95338dc10fb728f2/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:48.324701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.328665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb42c9-9be5-4729-bc58-49346ff1b323 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.387962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.409614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.410293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.410603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.410877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.411937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.414497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e72ceb-141e-44bc-a733-65914a010f90 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.414959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.421259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.422429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.423097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.423579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.426862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.429095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.453327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 991/2012] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:48.454444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.455063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1022/2013] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:20:48.461777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.477915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b24c-7a6b-4c3f-8989-97de4e7659b6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.547733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd21e8e-0136-486e-bd4e-964d2da5d13c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.553546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.554508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.554852 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.555123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] PATH_INFO: `/v3/projects/f337839d467943f7a383252f8214b0ec/users/0e6693ca9acb4f1f95338dc10fb728f2/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.561985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f337839d467943f7a383252f8214b0ec, user_id=0e6693ca9acb4f1f95338dc10fb728f2, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.565156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.576013 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.576965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.577366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.577797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.579044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 992/2014] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:48 2022] PUT /identity/v3/projects/f337839d467943f7a383252f8214b0ec/users/0e6693ca9acb4f1f95338dc10fb728f2/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:48.580032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.585989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.589815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.592079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c3aca9-b35b-40de-aff0-588cec2ef083 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.631990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1023/2015] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:48.633448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.633836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.634231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.677627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.687044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] MFA Rules not processed for user `0e6693ca9acb4f1f95338dc10fb728f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:48.688200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.702067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77c3064-b46c-457f-8487-d766474f7651 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.708866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.710123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.711048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.711144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] PATH_INFO: `/v3/users/14d255f9467140378ba82b051abb7e54` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.713142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d255f9467140378ba82b051abb7e54)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.715575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.770360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1024/2016] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:48.771270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.782442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146fd819-8072-4c97-8297-5602a3a6d4d5 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.793616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.793616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.793616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.793616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.793945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.793945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.797140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d255f9467140378ba82b051abb7e54'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.797943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d255f9467140378ba82b051abb7e54'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.798558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d255f9467140378ba82b051abb7e54'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.805397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1025/2017] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:48.805397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.805397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.806240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d255f9467140378ba82b051abb7e54'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.818704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252dd18-de03-4a52-bf89-4f18f0da85a6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.818913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d255f9467140378ba82b051abb7e54'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.823726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.824852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.825361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.825645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] PATH_INFO: `/v3/projects/5d8ef1b613ab423f82d6acda84c03c2e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.826837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8ef1b613ab423f82d6acda84c03c2e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.828780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:48.832183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 993/2018] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:48 2022] DELETE /identity/v3/users/14d255f9467140378ba82b051abb7e54 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:48.835482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.835482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.866030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.882186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8ef1b613ab423f82d6acda84c03c2e', 'request_id': 'req-3d865703-62a7-4f7d-9341-bb8e649a82b5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.888745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8ef1b613ab423f82d6acda84c03c2e', 'request_id': 'req-3d865703-62a7-4f7d-9341-bb8e649a82b5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.897087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8ef1b613ab423f82d6acda84c03c2e', 'request_id': 'req-3d865703-62a7-4f7d-9341-bb8e649a82b5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.897682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f6ce7-3fa8-4d51-b24a-809d0d36ed39 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.909402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] The token cache is being invalidate because project 5d8ef1b613ab423f82d6acda84c03c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:48.909688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.910344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:48.911910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1026/2019] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:48 2022] DELETE /identity/v3/projects/5d8ef1b613ab423f82d6acda84c03c2e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:48.912689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:48.914281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.952125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.952814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.953552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.953831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.955926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:48.987405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d865703-62a7-4f7d-9341-bb8e649a82b5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:48.992377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:48.993134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:48.993477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:48.993740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:48.993864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:48.994944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:48.996568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90370b87-0fa8-41af-80b4-d7664f802c05 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.008597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1027/2020] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:49.010532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.010889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.011224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.033932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.035429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.035575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:49.036830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.044970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1028/2021] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:49.045657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.045767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 994/2022] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:49.046546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.046955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.059639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5052a-6799-4231-8490-38011e772f41 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.069237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad049ca-0d3b-40b1-bb80-e1ca788be6e1 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.070617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.073036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.073622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.074724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] PATH_INFO: `/v3/users/579756abe1144b46b5e88345ab134bd5` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.077544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.078451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.078727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.079122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] PATH_INFO: `/v3/projects/d03eaae584e64d81af692442d5e1a117/users/cab00fd693c34d288e99bd271ac35551/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.084255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] RBAC: Authorizing `identity:delete_user(user_id=579756abe1144b46b5e88345ab134bd5)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.086868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.096989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d03eaae584e64d81af692442d5e1a117, user_id=cab00fd693c34d288e99bd271ac35551, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.102043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.129575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 995/2023] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:49 2022] PUT /identity/v3/projects/d03eaae584e64d81af692442d5e1a117/users/cab00fd693c34d288e99bd271ac35551/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:49.129575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.137921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.174228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '579756abe1144b46b5e88345ab134bd5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.174933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '579756abe1144b46b5e88345ab134bd5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.175563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '579756abe1144b46b5e88345ab134bd5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.189020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '579756abe1144b46b5e88345ab134bd5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.216541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-30c9f591-8bae-40fa-9697-5c504d328113 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '579756abe1144b46b5e88345ab134bd5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.226704 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581bf0e-4bb1-4a0f-bb07-cc0a04393749 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.230853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.231745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.232136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.232534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.234954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.250507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1029/2024] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:49 2022] DELETE /identity/v3/users/579756abe1144b46b5e88345ab134bd5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:49.251988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.252410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.252786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.285902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] MFA Rules not processed for user `cab00fd693c34d288e99bd271ac35551`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:49.287385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.299995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:49.346324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.348065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.360432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 996/2025] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:49.361453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.363411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.397774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1030/2026] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:49.409010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.410540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.446401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e282d41-87ef-4e0a-810b-d103f2cb1dd6 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.457401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:49.492638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2063225-8609-482d-8dbe-2026db0ca86d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.502025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.502962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.503358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.503705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.512703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.513289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.513517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.513790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.515845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.516877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.518706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.531515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1031/2027] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:49.607391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.609296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.622687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 997/2028] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:49.662210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:49.664257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:49.817613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29e5c-b9fe-4994-8be9-b7ad419ee7eb service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:49.826197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:49.827210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:49.827520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:49.829477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] PATH_INFO: `/v3/projects/8f14de31d23d4aef91d8df336bb2949c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:49.830975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f14de31d23d4aef91d8df336bb2949c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:49.832993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:49.915097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f14de31d23d4aef91d8df336bb2949c', 'request_id': 'req-c9cdeb80-68b6-4108-9cc0-88cb57967973'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.922868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:49.924397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f14de31d23d4aef91d8df336bb2949c', 'request_id': 'req-c9cdeb80-68b6-4108-9cc0-88cb57967973'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.940675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f14de31d23d4aef91d8df336bb2949c', 'request_id': 'req-c9cdeb80-68b6-4108-9cc0-88cb57967973'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.970450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] The token cache is being invalidate because project 8f14de31d23d4aef91d8df336bb2949c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:49.970850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.971568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:49.975014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1032/2029] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:49 2022] DELETE /identity/v3/projects/8f14de31d23d4aef91d8df336bb2949c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:49.979567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:50.016395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e240bcf-2a6a-43d1-ae8b-71a9c61b99ba service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:50.233369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:50.234374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.234656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.234928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] PATH_INFO: `/v3/users/601a06ce08b34323a78a9862b275b123` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:50.236184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601a06ce08b34323a78a9862b275b123)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:50.238337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:50.303808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601a06ce08b34323a78a9862b275b123'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.305643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601a06ce08b34323a78a9862b275b123'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.305822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601a06ce08b34323a78a9862b275b123'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.325362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:50.326637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601a06ce08b34323a78a9862b275b123'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.327445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:50.342156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:50.343759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601a06ce08b34323a78a9862b275b123'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.352836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cdeb80-68b6-4108-9cc0-88cb57967973 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:50.373962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:50.374950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.375178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.375454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] PATH_INFO: `/v3/projects/061f1577a1064ba88cdb94665b724762` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:50.376930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=061f1577a1064ba88cdb94665b724762)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:50.379753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:50.387555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 998/2030] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:49 2022] DELETE /identity/v3/users/601a06ce08b34323a78a9862b275b123 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:50.395385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:50.399348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:50.477587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061f1577a1064ba88cdb94665b724762', 'request_id': 'req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.491860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061f1577a1064ba88cdb94665b724762', 'request_id': 'req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.511397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061f1577a1064ba88cdb94665b724762', 'request_id': 'req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.514020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434bfe7-8bf5-4b61-8b8d-60a08b9fb042 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:50.525587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] The token cache is being invalidate because project 061f1577a1064ba88cdb94665b724762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:50.526356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.527308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.530281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1033/2031] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:50 2022] DELETE /identity/v3/projects/061f1577a1064ba88cdb94665b724762 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:50.534550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:50.537588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:50.591393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:50.592980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.592980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.593204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] PATH_INFO: `/v3/users/7cd32b5620144b43afd9b349c7054c3e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:50.594561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cd32b5620144b43afd9b349c7054c3e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:50.596725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:50.642889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ee68e-5aea-40fd-b65b-78bc5f842ab5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:50.648274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:50.649211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.649486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.649750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] PATH_INFO: `/v3/users/2a1833be5d8a47d9a4067da94794660f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:50.651607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1833be5d8a47d9a4067da94794660f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:50.654036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:50.671442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cd32b5620144b43afd9b349c7054c3e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.672582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cd32b5620144b43afd9b349c7054c3e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.673772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cd32b5620144b43afd9b349c7054c3e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.686520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cd32b5620144b43afd9b349c7054c3e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.719498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c49fe2ae-24b9-40b6-a27e-c3013d44bc4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cd32b5620144b43afd9b349c7054c3e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.740333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 999/2032] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:50 2022] DELETE /identity/v3/users/7cd32b5620144b43afd9b349c7054c3e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:50.743607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1833be5d8a47d9a4067da94794660f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.743607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1833be5d8a47d9a4067da94794660f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.743607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1833be5d8a47d9a4067da94794660f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.746204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.747400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.747966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:50.755467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1833be5d8a47d9a4067da94794660f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.766312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1833be5d8a47d9a4067da94794660f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:50.776588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:50.776588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:50.776965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:50.792147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1034/2033] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:50 2022] DELETE /identity/v3/users/2a1833be5d8a47d9a4067da94794660f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:50.799751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:50.881026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1000/2034] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:50.897689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:50.899185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:50.972472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996933d-89f9-452a-b3ca-153ee613278c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:50.977324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:50.978107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:50.978459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:50.979581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.076990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.079193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68922b3a-0fc1-47e1-b1f6-77fb7d072eba service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.087481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.089532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.095888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1035/2035] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:20:51.102725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3895b-94f2-4d84-abae-ee128704dda4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.128505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.129932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.131797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.132210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] PATH_INFO: `/v3/projects/9c68d40a41324e1d800ca35d304d2c4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.135258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c68d40a41324e1d800ca35d304d2c4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.139538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.250528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c68d40a41324e1d800ca35d304d2c4a', 'request_id': 'req-c7068411-3c22-4d50-9cdf-b5dcbb137e68'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.259196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c68d40a41324e1d800ca35d304d2c4a', 'request_id': 'req-c7068411-3c22-4d50-9cdf-b5dcbb137e68'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.274043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c68d40a41324e1d800ca35d304d2c4a', 'request_id': 'req-c7068411-3c22-4d50-9cdf-b5dcbb137e68'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.297955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] The token cache is being invalidate because project 9c68d40a41324e1d800ca35d304d2c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:51.298274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.299031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.304085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1001/2036] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:51 2022] DELETE /identity/v3/projects/9c68d40a41324e1d800ca35d304d2c4a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:51.310031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb594-d311-46ab-95b6-0a12a30e7fab None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.310642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb594-d311-46ab-95b6-0a12a30e7fab None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.311253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb594-d311-46ab-95b6-0a12a30e7fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.358372 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.359153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c48eb594-d311-46ab-95b6-0a12a30e7fab None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:51.359842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.360595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c48eb594-d311-46ab-95b6-0a12a30e7fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.508585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1036/2037] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:51.520089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.520266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.520567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.531278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7068411-3c22-4d50-9cdf-b5dcbb137e68 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.536976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.538048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.538445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.538940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] PATH_INFO: `/v3/projects/987a6c449ac84a8dbf1279e0169d0925` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.542186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] RBAC: Authorizing `identity:delete_project(project_id=987a6c449ac84a8dbf1279e0169d0925)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.544168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.555944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:51.558030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.568866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1037/2038] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:51.570059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.574316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.587577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d12e5-eea2-4e30-9ec3-0c0a72860819 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.594169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.595877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.596242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.596666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] PATH_INFO: `/v3/projects/031bd1ae0ca84ac09b2fc030436994f5` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.598027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] RBAC: Authorizing `identity:delete_project(project_id=031bd1ae0ca84ac09b2fc030436994f5)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.600020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.610212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:51.622707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987a6c449ac84a8dbf1279e0169d0925', 'request_id': 'req-bf73cf57-2b23-4560-8682-af1ea563502b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.631513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987a6c449ac84a8dbf1279e0169d0925', 'request_id': 'req-bf73cf57-2b23-4560-8682-af1ea563502b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.639119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:51.641183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987a6c449ac84a8dbf1279e0169d0925', 'request_id': 'req-bf73cf57-2b23-4560-8682-af1ea563502b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.656237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] The token cache is being invalidate because project 987a6c449ac84a8dbf1279e0169d0925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:51.656565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.657368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.662594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1002/2039] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:51 2022] DELETE /identity/v3/projects/987a6c449ac84a8dbf1279e0169d0925 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:51.662795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031bd1ae0ca84ac09b2fc030436994f5', 'request_id': 'req-bbcd9b05-a567-4a08-bd29-8d85428f309c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.662910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.662910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.669589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031bd1ae0ca84ac09b2fc030436994f5', 'request_id': 'req-bbcd9b05-a567-4a08-bd29-8d85428f309c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.678630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031bd1ae0ca84ac09b2fc030436994f5', 'request_id': 'req-bbcd9b05-a567-4a08-bd29-8d85428f309c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.691807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] The token cache is being invalidate because project 031bd1ae0ca84ac09b2fc030436994f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:51.692094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.692809 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bbcd9b05-a567-4a08-bd29-8d85428f309c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.694549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1038/2040] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:51 2022] DELETE /identity/v3/projects/031bd1ae0ca84ac09b2fc030436994f5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:51.696141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.696465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.696743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.715660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:51.716912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.745512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73cf57-2b23-4560-8682-af1ea563502b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.812279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1039/2041] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:51.812279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.813858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.827466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c33b2-fd7d-420f-8af0-55dd2f3cf255 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.833534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.834548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.834928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.835301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] PATH_INFO: `/v3/users/0e6693ca9acb4f1f95338dc10fb728f2` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.836746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e6693ca9acb4f1f95338dc10fb728f2)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.839650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.857093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.858391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.858819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.859605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.860722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:51.861827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.863513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b079-a723-4e28-8686-55414f26654d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.881981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1003/2042] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:51.882170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.882496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.882778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.906573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:51.908672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.925671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1004/2043] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:51.926700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:51.928085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:51.940601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeed46-aa13-4309-bbd2-9122a195e212 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:51.946340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:51.947012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:51.947811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:51.948083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:51.948944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:51.954710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e6693ca9acb4f1f95338dc10fb728f2'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.955647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e6693ca9acb4f1f95338dc10fb728f2'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.956491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e6693ca9acb4f1f95338dc10fb728f2'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.960397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:51.968763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e6693ca9acb4f1f95338dc10fb728f2'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:51.976832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e6693ca9acb4f1f95338dc10fb728f2'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.003963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1005/2045] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:51 2022] POST /identity/v3/projects => generated 381 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:52.003963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.004801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1040/2045] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:20:51 2022] DELETE /identity/v3/users/0e6693ca9acb4f1f95338dc10fb728f2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:52.005873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.007710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.016816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.065374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:52.116273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:52.126414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d87080-4eed-447b-ac68-951310787d1f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.152464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e475f4-8321-41e1-b8a3-62ae071e9fd6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.158293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.159189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.159483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.159782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.160625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:52.161120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.163231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.177752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1006/2046] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:52.184164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.198816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4228e2-1021-430e-96c0-b97738b331db admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.204840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.205619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.206004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.206399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.207311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.208652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.209869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.210008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.210425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.210850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.215095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.217368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.255636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1007/2047] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/projects => generated 409 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:52.256824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.261967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1041/2048] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:20:52.266026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.284104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b5c9d-6e98-45ff-a5b4-f266d8ec2b90 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.362331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.363228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.363948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.364431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.365278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.367368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.370409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d269c-d892-4a2d-936d-237163ad0a40 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.377262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.377960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.378402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.378571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.381918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.381918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.388037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1042/2049] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:52.389341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.392502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.410688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e12aa-30d2-4337-b8aa-ce666404dbc3 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.417535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.418657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.419896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.420381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] PATH_INFO: `/v3/projects/f337839d467943f7a383252f8214b0ec` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.424487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1008/2050] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/users => generated 305 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:52.424641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f337839d467943f7a383252f8214b0ec)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.424751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.424841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.434913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8787712-f2cc-4d92-a572-062587539741 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.439778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.440479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.440762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.441037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.441765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.443399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.455798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1009/2051] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:52.456734 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.472133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f6475-7cb3-4025-9db5-516a6297d440 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.489132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f337839d467943f7a383252f8214b0ec', 'request_id': 'req-a90c574d-cec9-46c4-a7c3-a63a150668dc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.495771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:52.498630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f337839d467943f7a383252f8214b0ec', 'request_id': 'req-a90c574d-cec9-46c4-a7c3-a63a150668dc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.509442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f337839d467943f7a383252f8214b0ec', 'request_id': 'req-a90c574d-cec9-46c4-a7c3-a63a150668dc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.525072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] The token cache is being invalidate because project f337839d467943f7a383252f8214b0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:20:52.525473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.526081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:20:52.528138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1043/2052] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:20:52 2022] DELETE /identity/v3/projects/f337839d467943f7a383252f8214b0ec => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:52.533172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.536116 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.550037 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.550721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.551006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.551397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.552043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.560783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f970e5-a0d5-4589-90e5-967081eee73d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.572562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1010/2053] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:52.574063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.574356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.574641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.598530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:52.600177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.617061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1011/2054] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:52.617061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.617061 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.626388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130810a5-a2ce-4d30-a5d4-dfb4fa25b01f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.631865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.632683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.633163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.633429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.634087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.635802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d5d21-056b-4c00-bcb6-14daf4af8ab6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.635922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c574d-cec9-46c4-a7c3-a63a150668dc admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.641939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.642968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.643349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.643726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] PATH_INFO: `/v3/projects/913d8fc8999b4cc2852b55d3127bfef8/users/561d31c350324eb88e86aadd21cb9aa6/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.658785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1012/2055] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:52.659714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] RBAC: Authorizing `identity:create_grant(project_id=913d8fc8999b4cc2852b55d3127bfef8, user_id=561d31c350324eb88e86aadd21cb9aa6, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.660582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b63c-c4f3-4bed-813a-463493214ce0 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.661035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b63c-c4f3-4bed-813a-463493214ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.661623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141b63c-c4f3-4bed-813a-463493214ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.663033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.678323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1044/2056] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:52 2022] PUT /identity/v3/projects/913d8fc8999b4cc2852b55d3127bfef8/users/561d31c350324eb88e86aadd21cb9aa6/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:52.679445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.693654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e141b63c-c4f3-4bed-813a-463493214ce0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:52.696859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e141b63c-c4f3-4bed-813a-463493214ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.702201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:52.754340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b216d-5b08-47ed-b672-83cfef68a06e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.759223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.760822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.761104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.761386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] PATH_INFO: `/v3/projects/d7d77be5267d47b88a05636949c27d74/users/483fd67d20e64bb9b8574ddc7b3c90d9/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.774680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1013/2057] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:52.776832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425c6cd-da95-458c-9344-24ad7647596c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.777140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425c6cd-da95-458c-9344-24ad7647596c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.777417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425c6cd-da95-458c-9344-24ad7647596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.787993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d77be5267d47b88a05636949c27d74, user_id=483fd67d20e64bb9b8574ddc7b3c90d9, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.790763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.807728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1045/2058] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:52 2022] PUT /identity/v3/projects/d7d77be5267d47b88a05636949c27d74/users/483fd67d20e64bb9b8574ddc7b3c90d9/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:52.807931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-6425c6cd-da95-458c-9344-24ad7647596c None None] MFA Rules not processed for user `561d31c350324eb88e86aadd21cb9aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:52.808518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.809195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6425c6cd-da95-458c-9344-24ad7647596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.869472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:52.884354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b8d5f-2a39-42f6-b849-91f32b911ae1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.885406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1014/2059] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:52.886537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.886823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.887099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.892749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.893416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.893697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.893972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.894707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:20:52.895014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.896718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.908956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1046/2060] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:20:52.909791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.915447 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] MFA Rules not processed for user `483fd67d20e64bb9b8574ddc7b3c90d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:52.916732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:52.919382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76019b3-6474-4e2a-867f-5c3dfbdb7d37 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:52.923978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:52.924650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:52.924932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:52.925226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:52.925897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:52.927780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:52.970224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1047/2061] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:52.971009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:52.997761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1015/2062] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:52.997761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.006830 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e35f76-30cc-4a6c-8a05-de00744c066f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.035169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.049428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2685f-234c-44b8-bebb-def02a6d23f4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.055584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.056344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.056765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.057107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.057874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.059862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.091966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.092188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1048/2063] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:53.092696 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.092981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.093217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.093340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.093940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.096134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.124163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1016/2064] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:52 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:53.124163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.135407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae3af-067e-4121-bba2-7b7ec8ad6263 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.182971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99690ba-c894-42c5-a98f-3f890c3d429f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.188538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.189367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.189711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.190291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.191051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.193076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.210118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.210957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.211335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.211622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.212448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.214969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.227118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1049/2065] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/projects => generated 407 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:53.228965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.240384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1017/2066] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:53.241968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.253448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62ae8f-be61-42de-aa96-1c05b00e6810 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.266658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.296641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfec86-c733-4942-a3fe-78bbb9a30754 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.301299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.302044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.302388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.302727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.303621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.305252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.318973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1050/2067] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.318973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.319281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.319958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.320238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.320537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.321249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.323685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.327028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06747161-e94c-493d-bc8f-5ad8d8460901 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.331852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.332527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.333032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.333211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.333850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.335484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.344337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1018/2068] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/users => generated 310 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:20:53.344337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.351873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1051/2069] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.351873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.354971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4113ee06-de03-4ea5-992f-f7738bb68fac admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.359842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.360743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.361021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.361291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] PATH_INFO: `/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e/users/5543834b95a64fc495db166faa7d117f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.361719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d6bcc-6cd3-4ca4-ac0c-5d54ae8971d9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.366368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.367006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.367290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.367569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.368285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.370493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.377087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70b608ddbf44ce6aeee9d08ace40e6e, user_id=5543834b95a64fc495db166faa7d117f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.379721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.381231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1052/2070] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.382027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.391953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec322122-a529-4e73-b48d-5cc597267e67 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.396025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1019/2071] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:53 2022] PUT /identity/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e/users/5543834b95a64fc495db166faa7d117f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:53.399631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.412803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.449293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.449966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.450248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.450521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.451247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.452881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.465808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1053/2072] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.465808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.466080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe8bafd-fcd4-41ed-b906-def90d82dd6f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.468642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.469532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.469843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.470115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] PATH_INFO: `/v3/projects/749318bc9bd34fe090b760738947c703/users/ea4aa7b8b53f4de8b324188758347f62/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.472660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66276f3f-d2f7-4475-8a9e-742be64f6117 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.477364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.477993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.478779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.479072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.479785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.481400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.486009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749318bc9bd34fe090b760738947c703, user_id=ea4aa7b8b53f4de8b324188758347f62, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.488756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.491690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1054/2073] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.492290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.503151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab57272-c37c-4401-8deb-9132c94bdfba admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.505433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1020/2074] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:53 2022] PUT /identity/v3/projects/749318bc9bd34fe090b760738947c703/users/ea4aa7b8b53f4de8b324188758347f62/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:53.508281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.576789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.577686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.577973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.578273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] PATH_INFO: `/v3/projects/e4b341a7554341bd88482be5ce5972a1/users/aa96ae271b884403892ea060532115e4/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.585362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95248cd1-4852-4a70-b14f-7ff8238b3b38 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.590500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.591381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.591757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.591936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] PATH_INFO: `/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e/users/5543834b95a64fc495db166faa7d117f/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.593744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b341a7554341bd88482be5ce5972a1, user_id=aa96ae271b884403892ea060532115e4, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.593900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f70b608ddbf44ce6aeee9d08ace40e6e, user_id=5543834b95a64fc495db166faa7d117f, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.596560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.596706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682581f-761b-4627-9edb-44debc211a7d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.608126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1021/2075] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:53 2022] PUT /identity/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e/users/5543834b95a64fc495db166faa7d117f/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:53.609155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.609940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1055/2076] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:53 2022] PUT /identity/v3/projects/e4b341a7554341bd88482be5ce5972a1/users/aa96ae271b884403892ea060532115e4/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:53.619933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c5fa-8301-4630-a004-1d09b03653b2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.620225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c5fa-8301-4630-a004-1d09b03653b2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.620526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9c5fa-8301-4630-a004-1d09b03653b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.644793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.647929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-dad9c5fa-8301-4630-a004-1d09b03653b2 None None] MFA Rules not processed for user `5543834b95a64fc495db166faa7d117f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:53.649805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dad9c5fa-8301-4630-a004-1d09b03653b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:53.676941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428488b-22b8-42a2-88e5-aa168c89d775 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.714412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.719197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1056/2077] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:20:53.723729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.724014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.724284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.742655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.743331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.743616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.744407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.745149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.746838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.750886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] MFA Rules not processed for user `aa96ae271b884403892ea060532115e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:53.752085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:53.756835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1022/2078] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:20:53.761344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.762956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:53.819673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1057/2079] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:53.820751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.821824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ad52c-5558-4d8c-b721-576ef9ee783d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.826669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.827316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.827596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.827878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.831894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c9dc-4d93-433a-945d-00ff5bb72c00 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.836461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.837337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.837609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.837705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.837894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] PATH_INFO: `/v3/projects/749318bc9bd34fe090b760738947c703/users/ea4aa7b8b53f4de8b324188758347f62/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.839139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.839931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=749318bc9bd34fe090b760738947c703, user_id=ea4aa7b8b53f4de8b324188758347f62, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.842679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fe7f6-ad16-446a-87c4-14effff5c071 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.848614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1023/2080] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:53.849402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.850718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:53.853647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1058/2081] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:20:53 2022] PUT /identity/v3/projects/749318bc9bd34fe090b760738947c703/users/ea4aa7b8b53f4de8b324188758347f62/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:20:53.864746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.865131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.865452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.866168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e8a59-4067-48fa-bc02-687ed9ff0b56 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.894471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] MFA Rules not processed for user `ea4aa7b8b53f4de8b324188758347f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:20:53.895677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:20:53.920203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:53.920894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:53.921155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:53.921428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:53.980584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:20:53.980760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:53.982100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:53.984868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1059/2082] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:20:53 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:20:53.986027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:53.992083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1024/2083] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:53.995955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcfa56-44ff-469d-9a49-b6ba8ae5eb21 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:53.999588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:54.000216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:54.000526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:54.000797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:54.057920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:54.059446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:54.068792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1060/2084] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:54.254813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:54.270575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eee0674-d5a7-424d-a28b-ada101ec792f service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:54.277587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:54.278507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:54.278846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:54.279129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:54.291842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:54.293721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:54.318227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1025/2085] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:20:54.520533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:20:54.530228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f33a59-cf49-44d7-8dac-92bd82307b4e service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:20:54.534397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:20:54.536046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:20:54.536441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:20:54.536727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:20:54.595801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:20:54.597258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:20:54.606698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1061/2086] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:20:54 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:00.913558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:00.935383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:00.986128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00387de8-be9f-428e-81b9-fcc1083c8896 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:00.992088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:00.993058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:00.993368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:00.993632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] PATH_INFO: `/v3/users/aa96ae271b884403892ea060532115e4` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:00.995049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa96ae271b884403892ea060532115e4)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:00.997145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.002328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.017748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedad6e-a494-4467-b2ed-540a5476e6fc service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.027905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.029475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.030154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.030395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] PATH_INFO: `/v3/users/561d31c350324eb88e86aadd21cb9aa6` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.032929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561d31c350324eb88e86aadd21cb9aa6)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:01.035525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.064175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa96ae271b884403892ea060532115e4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.065055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa96ae271b884403892ea060532115e4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.065836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa96ae271b884403892ea060532115e4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.087761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa96ae271b884403892ea060532115e4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.106861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ab73a71-534e-4439-b85e-899cedc818b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa96ae271b884403892ea060532115e4'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.117538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561d31c350324eb88e86aadd21cb9aa6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.118390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561d31c350324eb88e86aadd21cb9aa6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.119261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561d31c350324eb88e86aadd21cb9aa6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.133384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1026/2087] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:00 2022] DELETE /identity/v3/users/aa96ae271b884403892ea060532115e4 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:01.133748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561d31c350324eb88e86aadd21cb9aa6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.138778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.139076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.139357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.145484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1596e595-768b-49d2-b4dd-9d939cbbf7e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561d31c350324eb88e86aadd21cb9aa6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.159684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:01.161086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.179438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1062/2088] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:01 2022] DELETE /identity/v3/users/561d31c350324eb88e86aadd21cb9aa6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:01.186220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.186565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.187010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.211425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:01.213000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.218226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:01.278877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1027/2089] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:01.287714 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.289236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.296358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1063/2090] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:01.310232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.311993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.349110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:01.383751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183982c2-2b9f-4016-b44d-9d8ba9b0914e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.388688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.390369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.390664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.390943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.393820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a868c63-e9cf-49cc-bf21-7ec5b284b989 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.397729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.398389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.399242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.399524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.401166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:01.402637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.410088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:01.412880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322b898-2f70-4434-a9a9-ff5993cb08bb service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.413637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1028/2091] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:01.423531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1064/2092] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:01.635776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.637203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.650356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee0000-52b8-415e-96d1-b6ccb4885cfd service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.655311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.656184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.656528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.656760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] PATH_INFO: `/v3/users/cab00fd693c34d288e99bd271ac35551` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.657959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cab00fd693c34d288e99bd271ac35551)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:01.659963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.709975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cab00fd693c34d288e99bd271ac35551'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.710660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cab00fd693c34d288e99bd271ac35551'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.711269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cab00fd693c34d288e99bd271ac35551'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.718689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cab00fd693c34d288e99bd271ac35551'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.737285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cab00fd693c34d288e99bd271ac35551'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:01.756941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1029/2093] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:01 2022] DELETE /identity/v3/users/cab00fd693c34d288e99bd271ac35551 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:01.762206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.762570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.762946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.789292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:01.815214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.896680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1065/2094] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:01.907622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.909540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.969072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:01.970910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:01.977872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348dcb9-64bc-4768-846b-209e775c897f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.984907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.985651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.985877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef32994-c419-4e35-849a-89b8f163e5e2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:01.986065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.986513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.990737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:01.991720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:01.992257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:01.992583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] PATH_INFO: `/v3/projects/e4b341a7554341bd88482be5ce5972a1` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:01.994509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b341a7554341bd88482be5ce5972a1)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:01.997083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:01.999302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.001369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.016464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1030/2095] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:02.017467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.018768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.034240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ba7a0-b2cd-4f06-b9ba-4ef444f28abc service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.040908 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.041345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.041649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.042557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] PATH_INFO: `/v3/projects/913d8fc8999b4cc2852b55d3127bfef8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.043836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] RBAC: Authorizing `identity:delete_project(project_id=913d8fc8999b4cc2852b55d3127bfef8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.045853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.096455 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.098641 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b341a7554341bd88482be5ce5972a1', 'request_id': 'req-ad3ba31a-d377-4e65-b005-6f6d6b788540'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.108336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b341a7554341bd88482be5ce5972a1', 'request_id': 'req-ad3ba31a-d377-4e65-b005-6f6d6b788540'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.112183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '913d8fc8999b4cc2852b55d3127bfef8', 'request_id': 'req-1f1d7c4a-add7-4082-84bf-11341e47967e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.113837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.117953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b341a7554341bd88482be5ce5972a1', 'request_id': 'req-ad3ba31a-d377-4e65-b005-6f6d6b788540'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.119414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '913d8fc8999b4cc2852b55d3127bfef8', 'request_id': 'req-1f1d7c4a-add7-4082-84bf-11341e47967e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.130101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '913d8fc8999b4cc2852b55d3127bfef8', 'request_id': 'req-1f1d7c4a-add7-4082-84bf-11341e47967e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.132581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] The token cache is being invalidate because project e4b341a7554341bd88482be5ce5972a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:02.132871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.133567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ad3ba31a-d377-4e65-b005-6f6d6b788540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.135382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1066/2096] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:01 2022] DELETE /identity/v3/projects/e4b341a7554341bd88482be5ce5972a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:02.141125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9592f-b785-4b47-a3d3-18d99fca0a29 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.141535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9592f-b785-4b47-a3d3-18d99fca0a29 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.141903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9592f-b785-4b47-a3d3-18d99fca0a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.144834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] The token cache is being invalidate because project 913d8fc8999b4cc2852b55d3127bfef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:02.145124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.145784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.147942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1031/2097] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:02 2022] DELETE /identity/v3/projects/913d8fc8999b4cc2852b55d3127bfef8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:02.152679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.154912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.172633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-27a9592f-b785-4b47-a3d3-18d99fca0a29 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:02.173855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-27a9592f-b785-4b47-a3d3-18d99fca0a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.243692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d7c4a-add7-4082-84bf-11341e47967e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.248944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.249827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.250112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.250387 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] PATH_INFO: `/v3/users/5543834b95a64fc495db166faa7d117f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.251608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5543834b95a64fc495db166faa7d117f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.256341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1067/2098] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:02.256509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.261416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.261781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.262136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.286220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:02.287477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.296897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1068/2099] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:02.301019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.310375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6c267-e1ef-45e7-8340-28a42f73b8c2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.316880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.317542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.317824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.318097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.318859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:02.319166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.322336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.323576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5543834b95a64fc495db166faa7d117f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.324294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5543834b95a64fc495db166faa7d117f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.324952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5543834b95a64fc495db166faa7d117f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.333331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5543834b95a64fc495db166faa7d117f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.337553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1069/2100] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:21:02.343406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.346415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.351864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5543834b95a64fc495db166faa7d117f'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.362123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2045b-5ecf-4eb9-bfe2-64d54f89598e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.368705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.369849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.370133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.370444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] PATH_INFO: `/v3/projects/d03eaae584e64d81af692442d5e1a117` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.371523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1032/2101] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:02 2022] DELETE /identity/v3/users/5543834b95a64fc495db166faa7d117f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:02.373343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.373479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d03eaae584e64d81af692442d5e1a117)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.375103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.375596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.435034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd03eaae584e64d81af692442d5e1a117', 'request_id': 'req-479ca862-45e7-4e71-9f3a-2606a8169c8a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.442202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd03eaae584e64d81af692442d5e1a117', 'request_id': 'req-479ca862-45e7-4e71-9f3a-2606a8169c8a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.451226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.452460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd03eaae584e64d81af692442d5e1a117', 'request_id': 'req-479ca862-45e7-4e71-9f3a-2606a8169c8a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.456250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8aab5-cb75-4d1a-8ed1-50e70e31cf0e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.468994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] The token cache is being invalidate because project d03eaae584e64d81af692442d5e1a117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:02.470122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.471629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-479ca862-45e7-4e71-9f3a-2606a8169c8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.474774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1070/2102] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:02 2022] DELETE /identity/v3/projects/d03eaae584e64d81af692442d5e1a117 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:02.481331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6e38-6228-451a-91d6-9e28038e5310 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.481818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6e38-6228-451a-91d6-9e28038e5310 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.482166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c6e38-6228-451a-91d6-9e28038e5310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.504480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-653c6e38-6228-451a-91d6-9e28038e5310 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:02.505824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-653c6e38-6228-451a-91d6-9e28038e5310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.536042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.536802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.537085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.537357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.538051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.540268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.558166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.586057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1033/2103] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:02.588227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.590749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.606317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1071/2104] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:02.614026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.614318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.614586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.635240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:02.638600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.683664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a37a7a-7944-4919-826f-7a2e7e53e5ec admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.691695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.692502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.692746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.693026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.693725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.695759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.717040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1072/2105] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:02.718324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.719708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.722650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1034/2106] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/users => generated 283 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:21:02.723826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.733140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ac771-58a4-4bc7-afbb-b5a5d6bfc24d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.738938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.740023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.740572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.741536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] PATH_INFO: `/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.742602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f70b608ddbf44ce6aeee9d08ace40e6e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.744612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.745857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f6928f-5134-46ff-b388-eb9ce3233a6d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.752922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.753545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.753893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.754623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.755377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:02.755798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.757567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.771492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1035/2107] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:02.772268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.783533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08444c8f-42f6-4797-9bcd-ee8d5eb3ada5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.788683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.789365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.789654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.789930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.790656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.792974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.803854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1036/2108] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:02.807800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.808562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f70b608ddbf44ce6aeee9d08ace40e6e', 'request_id': 'req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.814994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f70b608ddbf44ce6aeee9d08ace40e6e', 'request_id': 'req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.818084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3b53c-761b-489b-903f-be0659dfc1b0 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.824529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f70b608ddbf44ce6aeee9d08ace40e6e', 'request_id': 'req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.847878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] The token cache is being invalidate because project f70b608ddbf44ce6aeee9d08ace40e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:02.848192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.848892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:02.854018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1073/2109] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:02 2022] DELETE /identity/v3/projects/f70b608ddbf44ce6aeee9d08ace40e6e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:02.854018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.854018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.894505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.917234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.918017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.918312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.918594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.919471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.922170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1b7e1-5d85-4217-95d9-ff3a8ac30ae9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.941237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e36a2ef-edae-4d77-92eb-9f3b4b038e4c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:02.948451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:02.949624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.950067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.951744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.952704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:02.954533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:02.957466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1037/2110] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:02.958888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:02.959230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:02.959513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:02.973147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:02.981838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1074/2111] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:02.984502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:02.989064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:02.995625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:03.000699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.015339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023446e9-9d1f-4364-b4ad-f5421b1a4c51 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.109134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.109949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.110355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.110665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.111583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.113989 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b86ba0-578f-498f-8c46-981aee090424 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.120788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1038/2112] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:03.121917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.142633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ea76b-c1f5-401d-8ed6-76aa2ecbda47 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.143854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1075/2113] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:02 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:03.145396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.145789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.146257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.150095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.151661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.151932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.152211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] PATH_INFO: `/v3/projects/eae15379454d4207b73229abbc4d23ac/users/c4dd4a24951b43d7a2e46af5c73d5486/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.167073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:03.168390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.172381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eae15379454d4207b73229abbc4d23ac, user_id=c4dd4a24951b43d7a2e46af5c73d5486, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.172381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.181089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1076/2114] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:03.182133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.190759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1039/2115] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:03 2022] PUT /identity/v3/projects/eae15379454d4207b73229abbc4d23ac/users/c4dd4a24951b43d7a2e46af5c73d5486/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:03.191804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.192115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567ef4f-b395-4037-ba68-947e8503cd66 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.193514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.252714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.253388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.253669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.253943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.254664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.256369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.262288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd78b3-8515-4ef1-b2c2-eca10e3b1262 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.269684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1077/2116] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:03.269684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.270609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.271494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.271782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.272056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] PATH_INFO: `/v3/users/483fd67d20e64bb9b8574ddc7b3c90d9` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.273310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=483fd67d20e64bb9b8574ddc7b3c90d9)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.275264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.281135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893c121-e1e5-4ec4-9ccd-9bea633f6189 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.286448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.287122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.287399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.287680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.288440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:03.288755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.290395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202bf54-8c42-4a18-9ef2-e016e4699312 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.304691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1078/2117] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:03.305993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.306287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.306544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.336728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] MFA Rules not processed for user `c4dd4a24951b43d7a2e46af5c73d5486`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:03.337998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.339341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '483fd67d20e64bb9b8574ddc7b3c90d9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.340122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '483fd67d20e64bb9b8574ddc7b3c90d9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.340757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '483fd67d20e64bb9b8574ddc7b3c90d9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.350230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '483fd67d20e64bb9b8574ddc7b3c90d9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.352665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:03.368900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '483fd67d20e64bb9b8574ddc7b3c90d9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.387040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1040/2118] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:03 2022] DELETE /identity/v3/users/483fd67d20e64bb9b8574ddc7b3c90d9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:03.390148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.391653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.408409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:03.424449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1079/2119] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:03.425800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.427924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.443443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114ce80-6e7b-4ff4-8d5f-e2cdb10529b9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.472097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9428a4bc-a090-4b02-8afb-de283bb4af4c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.478207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.478911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.479731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.480018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.480896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.482449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816efbeb-c7ef-4c96-9d91-8ccc5ba3faf8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.494435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1041/2120] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:03.495946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76604-a8da-4f78-8ed8-c30ae391aa7e None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.496245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76604-a8da-4f78-8ed8-c30ae391aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.496593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf76604-a8da-4f78-8ed8-c30ae391aa7e None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.498093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1042/2121] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:21:03.499334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.499624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.500519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.507817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.508689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.509072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.509445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.510243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.512334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.520324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:03.522088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.532783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1043/2122] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:03.533658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.534961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.548704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1080/2123] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:03.549668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.608338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35bc1b-1233-4921-9215-57044de44bd7 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.621181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142daf45-e4a6-4b3a-9f10-8e2e6f9b13e5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.626327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.627228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.627508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.627784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] PATH_INFO: `/v3/projects/09a8ff92fd184f9e988166698e16e034/users/7d643d155d8f44179c0b5410ddde038d/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.646000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a8ff92fd184f9e988166698e16e034, user_id=7d643d155d8f44179c0b5410ddde038d, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.648710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.662494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1081/2124] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:03 2022] PUT /identity/v3/projects/09a8ff92fd184f9e988166698e16e034/users/7d643d155d8f44179c0b5410ddde038d/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:03.662656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.663381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.664992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.669184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.669525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.669941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.705360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:03.722970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6da988-f332-4f26-b49c-73957f3ed73b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.726614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.727262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.727542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.727813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.731412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.733252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.742294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1044/2125] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:03.743113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.747650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:03.754441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472220f5-9975-4239-a4c9-107adce6a0e8 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.805023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.807273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f287859b-0940-4a14-86f1-e7a329fe2b26 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.814732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.815402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.815683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.815957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.816684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.818612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.820207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1082/2126] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:03.822346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a396e9-0c33-4269-a394-b8a001205432 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.823693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a396e9-0c33-4269-a394-b8a001205432 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.824201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a396e9-0c33-4269-a394-b8a001205432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.846176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-71a396e9-0c33-4269-a394-b8a001205432 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:03.847685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-71a396e9-0c33-4269-a394-b8a001205432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.859023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1045/2127] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:03.860656 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:03.864917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:03.880334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01748dfd-da4a-4f2b-bac1-f112ee690cce admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:03.892192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:03.895095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.895513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.896517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] PATH_INFO: `/v3/users/0dc17670edbb4cc797103b0d41835bf9` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.899560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dc17670edbb4cc797103b0d41835bf9)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:03.906942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:03.950159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1083/2128] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:03.952793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:03.953293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:03.953759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:03.975300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dc17670edbb4cc797103b0d41835bf9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.976674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dc17670edbb4cc797103b0d41835bf9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.977477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dc17670edbb4cc797103b0d41835bf9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:03.987420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dc17670edbb4cc797103b0d41835bf9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.001549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] MFA Rules not processed for user `7d643d155d8f44179c0b5410ddde038d`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:04.003370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.005592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dc17670edbb4cc797103b0d41835bf9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.037832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1046/2129] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:03 2022] DELETE /identity/v3/users/0dc17670edbb4cc797103b0d41835bf9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:04.039019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.041483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.096288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1084/2130] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:04.097312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.098620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.136625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a970a-f896-4464-b88f-bcd26bd72f8d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.156781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.205401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82274e9b-8d6a-48d0-8c75-23ba77255ecc admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.219416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.222182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.222523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.223635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.224507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.226023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.230907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.261095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1047/2131] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:04.262222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.264804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.273876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.274962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.275267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.275750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.279086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.279593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2dc56-c47e-4954-baf6-293cfd55680a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.286710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.287444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.287698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.288038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.289048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.290637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.297255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.299208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.311546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1085/2132] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:04.312549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.328072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf010b7-fc47-4480-b851-93229c53dd97 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.334393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.335364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.335761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.336082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.337119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.340104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.356012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1086/2133] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:04.357457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.364414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1048/2134] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:04 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:04.365476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.367283 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.388811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90119d-5788-4793-bfeb-344edcdb054e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.447498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25987a1a-20e5-48c5-97f9-3727eaaff76b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.452202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.453137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.453416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.453720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.453922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] PATH_INFO: `/v3/projects/c4dd7eb571234653afc5196558ad6e8a/users/fed4f0efd3dc4575a5c6eafe0e0e4e66/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.454592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.454883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.455185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] PATH_INFO: `/v3/projects/d7d77be5267d47b88a05636949c27d74` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.456371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d77be5267d47b88a05636949c27d74)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.458294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.469261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4dd7eb571234653afc5196558ad6e8a, user_id=fed4f0efd3dc4575a5c6eafe0e0e4e66, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.471960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.489536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1049/2135] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:04 2022] PUT /identity/v3/projects/c4dd7eb571234653afc5196558ad6e8a/users/fed4f0efd3dc4575a5c6eafe0e0e4e66/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:04.490383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.492004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.514728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d77be5267d47b88a05636949c27d74', 'request_id': 'req-bdaa2471-53f7-4630-ae12-4469b5ea7bff'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.520167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d77be5267d47b88a05636949c27d74', 'request_id': 'req-bdaa2471-53f7-4630-ae12-4469b5ea7bff'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.528534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d77be5267d47b88a05636949c27d74', 'request_id': 'req-bdaa2471-53f7-4630-ae12-4469b5ea7bff'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.540340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] The token cache is being invalidate because project d7d77be5267d47b88a05636949c27d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:04.540561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.541252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.542978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1087/2136] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:04 2022] DELETE /identity/v3/projects/d7d77be5267d47b88a05636949c27d74 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:04.543801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.545125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.563546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80ee4d7-bc23-4b03-abdb-eb17acd72386 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.597521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.606316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.615475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa2471-53f7-4630-ae12-4469b5ea7bff admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.620259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.620972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.621261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.621538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.622233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.624164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c622ab-0c89-4475-8c1c-5e69a3aae9a9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.624743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.625599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.625880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.626151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] PATH_INFO: `/v3/projects/b175a657eab24493b2adbec405cbd515` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.627319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b175a657eab24493b2adbec405cbd515)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.629268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.646391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1088/2137] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:04 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:04.647670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.647958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.648234 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.675450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] MFA Rules not processed for user `fed4f0efd3dc4575a5c6eafe0e0e4e66`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:04.676695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.681107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b175a657eab24493b2adbec405cbd515', 'request_id': 'req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.686862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b175a657eab24493b2adbec405cbd515', 'request_id': 'req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.694410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b175a657eab24493b2adbec405cbd515', 'request_id': 'req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.707901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] The token cache is being invalidate because project b175a657eab24493b2adbec405cbd515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:04.708184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.708926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.710468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1050/2138] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:04 2022] DELETE /identity/v3/projects/b175a657eab24493b2adbec405cbd515 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:04.711299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.715499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.743045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1089/2139] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:04.743884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.745508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.791674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfe02c-95ae-4526-8ae2-c4a5e89ea6da admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.796294 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.797196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.797466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.803543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] PATH_INFO: `/v3/users/ea4aa7b8b53f4de8b324188758347f62` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.805353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4aa7b8b53f4de8b324188758347f62)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.807529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.824261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef31a3-6d56-4e48-8c25-4034d8ea6e1e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.829077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.829759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.830427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.830427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.831120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.832772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d693f-e23f-4cef-b569-ac543dfd4c1b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:04.849678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1090/2140] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:04.849678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.849678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.849678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.862149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4aa7b8b53f4de8b324188758347f62'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.863557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4aa7b8b53f4de8b324188758347f62'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.864415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4aa7b8b53f4de8b324188758347f62'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.868369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:04.870124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.873539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4aa7b8b53f4de8b324188758347f62'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.878983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1091/2141] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:04.879791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.881534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.887093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4aa7b8b53f4de8b324188758347f62'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:04.888906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.903255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1051/2142] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:04 2022] DELETE /identity/v3/users/ea4aa7b8b53f4de8b324188758347f62 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:04.904066 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:04.905471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:04.946813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df906fe-6709-47b1-9ad0-b0bfe839cef6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.957649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:04.981953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10b36d-773c-4380-b7a8-94a203917fef admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:04.988442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:04.989239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:04.989619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:04.989994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:04.990833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:04.993123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f19ad-45fa-4954-8f56-d44f64a92e93 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.008081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1052/2143] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:05.008821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.008821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.008821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.009098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.009181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.009276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.009276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.009425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.029222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:05.030088 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.043773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1053/2144] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:05.044792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.055257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf706cf-0bca-4551-92ee-cd265b537268 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.060162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.061100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.061430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.061737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] PATH_INFO: `/v3/projects/f7985cfd55bf43ce9038a87d9074209c/users/7d33b112503c41ec97d9ebc8f2ee1ac6/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.079180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7985cfd55bf43ce9038a87d9074209c, user_id=7d33b112503c41ec97d9ebc8f2ee1ac6, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.082245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.094109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.095558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46742ed4-cd39-481c-9a99-7bd568098a6c service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.103843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1054/2145] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:05 2022] PUT /identity/v3/projects/f7985cfd55bf43ce9038a87d9074209c/users/7d33b112503c41ec97d9ebc8f2ee1ac6/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:05.105849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.110207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1092/2146] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:05.115913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.116202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.116512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.150329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] MFA Rules not processed for user `7d33b112503c41ec97d9ebc8f2ee1ac6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:05.152336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.187570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10726258-63b8-4dc0-a9ff-a0f8b9ed1d39 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.192570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.193292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.193574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.193843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.194741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:05.195050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.196726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.218684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1055/2147] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:05.219605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.222980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.242057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7c07f-e804-423c-aadd-929884106536 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.242337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1093/2148] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:05.242337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.247186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.248684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.248684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.249067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] PATH_INFO: `/v3/users/fed4f0efd3dc4575a5c6eafe0e0e4e66` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.250408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed4f0efd3dc4575a5c6eafe0e0e4e66)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.253249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.257064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ddb9d-075c-43b1-9370-b5d7aaf41588 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.263696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.264686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.265195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.265752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.266843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.269140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.302151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1094/2149] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:05.303142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.305960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.316075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:05.320116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed4f0efd3dc4575a5c6eafe0e0e4e66'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.331756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed4f0efd3dc4575a5c6eafe0e0e4e66'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.331756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed4f0efd3dc4575a5c6eafe0e0e4e66'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.334759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed4f0efd3dc4575a5c6eafe0e0e4e66'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.349092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed4f0efd3dc4575a5c6eafe0e0e4e66'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.356376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:05.366158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1056/2150] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:05 2022] DELETE /identity/v3/users/fed4f0efd3dc4575a5c6eafe0e0e4e66 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:05.369787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.369787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.399945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7be320-fe77-41ab-a32c-6a6fd2fc24ba admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.455239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9608ed9-4f1b-4540-ad8b-7bc62fe82aff admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.460580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.461377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.461660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.461933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.462626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.464689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.478692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.480165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.480656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.481618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] PATH_INFO: `/v3/projects/749318bc9bd34fe090b760738947c703` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.483948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749318bc9bd34fe090b760738947c703)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.486895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.516999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1057/2151] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:05.516999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.521833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.560068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749318bc9bd34fe090b760738947c703', 'request_id': 'req-174e6a8b-34af-43de-bad3-47ac00667155'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.568207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749318bc9bd34fe090b760738947c703', 'request_id': 'req-174e6a8b-34af-43de-bad3-47ac00667155'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.581430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749318bc9bd34fe090b760738947c703', 'request_id': 'req-174e6a8b-34af-43de-bad3-47ac00667155'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.598700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] The token cache is being invalidate because project 749318bc9bd34fe090b760738947c703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:05.598991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.599813 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-174e6a8b-34af-43de-bad3-47ac00667155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:05.601790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1095/2152] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:05 2022] DELETE /identity/v3/projects/749318bc9bd34fe090b760738947c703 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:05.606924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.607213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.607488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.617025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123254-44ea-47c4-a57f-9487add01e0b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.627595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:05.628820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.679558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:05.688212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.689106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.689766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.690145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.691117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.693857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f716db-9523-45c7-a1cb-f2611af8ecc4 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.695560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1096/2153] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:05.697128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.704151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.718213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699d90-da97-4d8e-95d3-d6987b3a7ec5 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.721871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1058/2154] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:05.723228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.723670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.723971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.728686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.730052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.730403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.732632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.733690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.735726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.744168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:05.745672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.753718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1059/2155] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:05.754465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.763558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.764186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:05.779540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1097/2156] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/users => generated 295 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:05.786940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.794413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.822112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e92531-e6eb-462c-abb2-a2193a544f81 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.828846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.829534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.829823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.830076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.830808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.832492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca130b9-9af6-48ca-9829-dc8dc43114b0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.845187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1098/2157] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:05.849623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.849914 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.850222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.873504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:05.874707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:05.878321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3786-48d7-4fe2-b413-5aa644014493 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.882738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1099/2158] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:05.884368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.886165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.886758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.887127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.887330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.897917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f2873-1b6c-411d-ae6d-f17410ebea4b None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.902852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.903504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.903785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.904058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.904721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.904919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.907094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.907282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.917552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1100/2159] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:05.918424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.925303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1060/2160] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:05.927041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.928362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c77c69-6de5-4226-8658-376fddd34e8d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.933841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.934490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.934768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.935040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.935764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.937409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.944703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0290e-0b95-4e2f-a474-635f9674827d service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.950586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1101/2161] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:05.953820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.954497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.954964 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.955466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.955947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.956737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:05.957048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:05.958674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:05.965308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df416e0-a37e-4a06-9cfe-f23a1d6c2722 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.970671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.971300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1061/2162] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:05.971933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:05.972835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.973190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.973535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] PATH_INFO: `/v3/projects/2ed3c09a34a04ebe826fc93487a307cf/users/c86df20b1502499d905803f5b491013b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.982258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e1aad-fbed-44e2-bb29-887a5df59bc8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:05.987881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:05.988799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:05.989427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:05.989427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] PATH_INFO: `/v3/projects/8e9c8b24824e4336b52f0ce5dfd84d81/users/b9b3f3fbf046451a9ec031844f8c35f6/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:05.997819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed3c09a34a04ebe826fc93487a307cf, user_id=c86df20b1502499d905803f5b491013b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.000981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.010535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e9c8b24824e4336b52f0ce5dfd84d81, user_id=b9b3f3fbf046451a9ec031844f8c35f6, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.013359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.017644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1102/2163] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:05 2022] PUT /identity/v3/projects/2ed3c09a34a04ebe826fc93487a307cf/users/c86df20b1502499d905803f5b491013b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.021540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.029651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1062/2164] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:05 2022] PUT /identity/v3/projects/8e9c8b24824e4336b52f0ce5dfd84d81/users/b9b3f3fbf046451a9ec031844f8c35f6/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.030703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.103227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1762dc-28a7-4f7e-b753-a0537e69b645 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.104849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca7a25c-dd84-4025-b3ce-45baf4caa882 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.113050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.113834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.113969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.114172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.114451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.114657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.114985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.115150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.115251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.115985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.117836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f4de8-5957-4722-9c65-d99c2ef6a6c7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.118299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.132664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1063/2165] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:06.132664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.132664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.132664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.153023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1103/2166] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:06.155471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.163284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] MFA Rules not processed for user `b9b3f3fbf046451a9ec031844f8c35f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:06.164589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.196732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:06.230395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb93fd5-72d1-4aa9-b4b3-95cc09674c7e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.237029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1064/2167] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:06.238092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.240685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.241711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.242105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.242513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] PATH_INFO: `/v3/projects/2ed3c09a34a04ebe826fc93487a307cf/users/c86df20b1502499d905803f5b491013b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.245485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed3c09a34a04ebe826fc93487a307cf, user_id=c86df20b1502499d905803f5b491013b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.248516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.250174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d614c08-aa80-4631-a192-575b36aec6dc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.256077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.256776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.257053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.257328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.258097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.260274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.260969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1104/2168] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:06 2022] PUT /identity/v3/projects/2ed3c09a34a04ebe826fc93487a307cf/users/c86df20b1502499d905803f5b491013b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.261640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.276597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:06.288457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1065/2169] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:06.289561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.292552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.306625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42020e8-ba78-44a2-90cc-e2cb03519d28 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.330199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa2015-fc0c-44e8-8c57-e653017e9e62 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.335150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.335969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.336254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.336565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.337378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.339875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4289278-13c1-48d0-baca-a488f1bcf01c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.369390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1105/2170] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:06.369390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.369390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.369390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.371305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.372228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.372571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.372848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] PATH_INFO: `/v3/projects/c4dd7eb571234653afc5196558ad6e8a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.374024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4dd7eb571234653afc5196558ad6e8a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.375965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.394796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] MFA Rules not processed for user `c86df20b1502499d905803f5b491013b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:06.395996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.427081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4dd7eb571234653afc5196558ad6e8a', 'request_id': 'req-020066d9-7f20-46cf-9f3a-7a4cadd51800'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:06.432800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4dd7eb571234653afc5196558ad6e8a', 'request_id': 'req-020066d9-7f20-46cf-9f3a-7a4cadd51800'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:06.440180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4dd7eb571234653afc5196558ad6e8a', 'request_id': 'req-020066d9-7f20-46cf-9f3a-7a4cadd51800'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:06.452089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] The token cache is being invalidate because project c4dd7eb571234653afc5196558ad6e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:06.452404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:06.453188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:06.455012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1066/2171] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:06 2022] DELETE /identity/v3/projects/c4dd7eb571234653afc5196558ad6e8a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.455808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.459501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.474356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1106/2172] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:06.475178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.476500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.508307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:06.531030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020066d9-7f20-46cf-9f3a-7a4cadd51800 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.535857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.536557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.536843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.537124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.537850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.539469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c30e85-4628-4f58-8c25-91853982be59 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.542894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6abf62-56fb-4cb8-96a0-77db506d7aca None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.547548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.548209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.548513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.548786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.549466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.549968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1067/2173] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:06.551220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.551453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.551545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.551778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.570054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:06.571267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.572038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1107/2174] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:06.572934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.576414 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.582412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1068/2175] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:06.583067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.591851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:06.593796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a66aa-5660-47be-90a3-f897461032a9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.598648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.599303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.599583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.599856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.600625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.602217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.613705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1069/2176] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:06.614383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.625921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe4ba8-9ec6-4b9d-b5bf-cd4eb6ba08b5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.630679 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.631340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.631623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.631900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.632731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.634341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44fbdf-dade-41a2-80b6-40991e989440 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.642982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e23eec-fe53-4be3-a13c-6059fb68f67a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.644921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1070/2177] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:06.646171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.646463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.646743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.647328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.648367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.648709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.649055 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.651538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.675276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:06.676578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.688522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1071/2178] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:06.688522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.699855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6c063-dd02-48e0-8348-8b396b4d299f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.704722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.705812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.706337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.706661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] PATH_INFO: `/v3/projects/29dba7f4f5ee421f83ddf9c355e545c4/users/613645639a4143c0a66184c85708ec5b/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.722624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29dba7f4f5ee421f83ddf9c355e545c4, user_id=613645639a4143c0a66184c85708ec5b, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.723868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.725396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.725684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.737526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1108/2179] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:06.738341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.742520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1072/2180] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:06 2022] PUT /identity/v3/projects/29dba7f4f5ee421f83ddf9c355e545c4/users/613645639a4143c0a66184c85708ec5b/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.743340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:06.758945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48edc2d-63af-44e5-8a40-79e859fe750b service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.839820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48f5ad-6db8-49d3-8741-a595cb4d339c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:06.848808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.850055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.850343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.850793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.851806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:06.852128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.854759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a4230-876d-4b5f-8fb1-2024289ed290 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.872249 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1073/2181] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:06.874012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.874450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.874907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.877298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:06.879081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:06.879550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:06.879980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] PATH_INFO: `/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3/users/b7dd288c0b974125b98b772f253aa3db/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:06.911284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc3201d4ef40c0a836c9b7d620b0c3, user_id=b7dd288c0b974125b98b772f253aa3db, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:06.915096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:06.922507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] MFA Rules not processed for user `613645639a4143c0a66184c85708ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:06.925293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:06.941101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1109/2182] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:06 2022] PUT /identity/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3/users/b7dd288c0b974125b98b772f253aa3db/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:06.941437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.009335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.029146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1074/2183] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:07.029627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb318f-a536-4ae2-a92e-b60401d1aff3 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.030628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.035768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.036503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.037110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.037412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.038121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.039063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.040148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.046662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2a016-3bec-4ab6-840d-3f39bf972430 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.054766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.055680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.056416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.057008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.058578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.061035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.077049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1110/2184] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:06 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:07.078069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.101198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1075/2185] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:07.103777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.128503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68f748-66ab-4b62-b328-80185fc92539 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.159777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5b1d6-ee39-4ca6-8dae-790b275a9c88 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.166038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.166876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.167164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.167445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.168149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.170125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.198983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1111/2186] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:07.200244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.201046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.201431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.201808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.202629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.202872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.204793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.225180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1076/2187] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:07.226083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.235898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57678da-cc77-4ed7-9c38-b6edf25a5cc3 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.266227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4da41-2114-4304-ac6a-4e5e125a777b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.272069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.273111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.273403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.273676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] PATH_INFO: `/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3/users/b7dd288c0b974125b98b772f253aa3db/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.275697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=40dc3201d4ef40c0a836c9b7d620b0c3, user_id=b7dd288c0b974125b98b772f253aa3db, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.278415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.289648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1112/2188] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:07 2022] PUT /identity/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3/users/b7dd288c0b974125b98b772f253aa3db/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:07.290493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.297660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.298534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.299081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.299362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.300083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.302047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff9b9-4047-4736-8356-0d812cd22aad admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.326526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1077/2189] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:07.326526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.326526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.326526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.347650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.351558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] MFA Rules not processed for user `b7dd288c0b974125b98b772f253aa3db`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:07.352940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.360638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2395efd-153d-4de1-9951-2dc7e3f341dc admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.365415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.366077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.366364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.366645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.367376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.369058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.379456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1113/2190] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:07.380250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.390331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cdacf-07ee-4af2-939e-50f7f7d4bb57 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.394921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.395549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.395830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.396109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.396856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.398422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.409224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1114/2191] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:07.410076 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.416624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.419597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3a22d-cac6-4d7a-9e59-9fb25eec5e7d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.424571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.425213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.425498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.425773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.426492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.427534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1078/2192] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:07.428073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.428357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.438949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1115/2193] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:07.439199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc249bc-0297-40a2-8c00-dad837936b72 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.442680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.444073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.445137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.445533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.445813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] PATH_INFO: `/v3/projects/a188de6d192d4ed79e9bbe32dbac455d/users/26fe6ad7191047c68abee5322ca0e58b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.461186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188de6d192d4ed79e9bbe32dbac455d, user_id=26fe6ad7191047c68abee5322ca0e58b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.463899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.477419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1079/2194] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:07 2022] PUT /identity/v3/projects/a188de6d192d4ed79e9bbe32dbac455d/users/26fe6ad7191047c68abee5322ca0e58b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:07.481036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.501514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513f3d5-35fb-4118-bca3-e08a04754d99 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.543255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf578d-f30d-456e-9c51-320cb4fc9bda admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.548024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.548976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.549262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.549538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] PATH_INFO: `/v3/projects/667d4eb3be234d729065069069b06fd4/users/aac74f78cfa64c32be3f495125ff8d59/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.549941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.550671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.551066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.551415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.565746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] RBAC: Authorizing `identity:create_grant(project_id=667d4eb3be234d729065069069b06fd4, user_id=aac74f78cfa64c32be3f495125ff8d59, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.568529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.583340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1080/2195] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:07 2022] PUT /identity/v3/projects/667d4eb3be234d729065069069b06fd4/users/aac74f78cfa64c32be3f495125ff8d59/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:07.584391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.618843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.620996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2d9b2-02ef-414d-beb4-e2718592d3f4 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.631107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1116/2196] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:07.632515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c394da-09f9-456f-8091-2fd43712dd3d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.632991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c394da-09f9-456f-8091-2fd43712dd3d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.633277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c394da-09f9-456f-8091-2fd43712dd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.656932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27f7e-51d2-4d5c-a019-a3e8b4fc294d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.667141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b5c394da-09f9-456f-8091-2fd43712dd3d None None] MFA Rules not processed for user `aac74f78cfa64c32be3f495125ff8d59`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:07.667421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.667421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.667421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.667688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b5c394da-09f9-456f-8091-2fd43712dd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.667782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.667782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.667782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.678343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1081/2197] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:07.682772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.691676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.696674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441ec76-c9b0-4490-a444-85a6b73cf086 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.703056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.704280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.704781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.705716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] PATH_INFO: `/v3/projects/a188de6d192d4ed79e9bbe32dbac455d/users/26fe6ad7191047c68abee5322ca0e58b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.707857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a188de6d192d4ed79e9bbe32dbac455d, user_id=26fe6ad7191047c68abee5322ca0e58b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.710578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.721076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1082/2198] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:07 2022] PUT /identity/v3/projects/a188de6d192d4ed79e9bbe32dbac455d/users/26fe6ad7191047c68abee5322ca0e58b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:07.721854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.748123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1117/2199] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:07.755027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.755027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.755270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.775506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:07.785794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] MFA Rules not processed for user `26fe6ad7191047c68abee5322ca0e58b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:07.787025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.791138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f087-24c3-4e17-a638-36013b78b277 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.795168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.795817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.796098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.796400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.798237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.868818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.871062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.877740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1118/2200] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:07.878607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.880120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.888713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1083/2201] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:07.890641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:07.892705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:07.907140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78c9eb-dd38-4b20-a8ae-b542c24e23b2 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.911437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.912445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.912771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.912939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:07.924930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:07.927321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:07.940686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1084/2202] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:07.958359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265264fd-97e1-43de-af9a-b05d72833118 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:07.964678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:07.965628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:07.965792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:07.966107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:08.051342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:08.054667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:08.072496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1119/2203] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:07 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:08.587800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:08.597449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b77ccc-2672-4ddd-ba0b-5be312953d28 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:08.601532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:08.602337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:08.602636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:08.602915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:08.605356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:08.673382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:08.674837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dde809-1d9d-4ef7-b4b4-f9139a961d1e service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:08.683813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1085/2204] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:08.793017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:08.797413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:08.884371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e7e09-e63c-45af-b5af-aa03a41d9b26 service glance] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:08.890072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:08.891080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:08.891311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:08.891631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] PATH_INFO: `/v3/users/613645639a4143c0a66184c85708ec5b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:08.893041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=613645639a4143c0a66184c85708ec5b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:08.894963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:08.947415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '613645639a4143c0a66184c85708ec5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:08.948239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '613645639a4143c0a66184c85708ec5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:08.949085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '613645639a4143c0a66184c85708ec5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:08.957221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '613645639a4143c0a66184c85708ec5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:08.965643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '613645639a4143c0a66184c85708ec5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:08.983538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1120/2205] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:08 2022] DELETE /identity/v3/users/613645639a4143c0a66184c85708ec5b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:08.987956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:08.988251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:08.988563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:09.005033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:09.006220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:09.090080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1086/2206] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:09.101903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:09.105863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:09.205468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18bd8b1-24ef-4476-bec6-a44baf56d89f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:09.210410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:09.211090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:09.211359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:09.211638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:09.223211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:09.228838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:09.240912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1121/2207] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:09.430087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:09.432800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:09.494706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:09.539572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f86a-c92c-424b-bd38-ee2a7a671745 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:09.544533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:09.545369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:09.545649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:09.546044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:09.551087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:09.635466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:09.637441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:09.654949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1087/2208] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:09.657097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:09.658982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:09.673420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65375f9-89c9-4f80-a894-f1b840424822 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:09.680770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:09.681639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:09.683864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:09.683864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] PATH_INFO: `/v3/projects/29dba7f4f5ee421f83ddf9c355e545c4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:09.684898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=29dba7f4f5ee421f83ddf9c355e545c4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:09.687421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:09.721200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:09.758553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29dba7f4f5ee421f83ddf9c355e545c4', 'request_id': 'req-75f3508a-a73e-4c14-9edd-25ad2b4275fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.765304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29dba7f4f5ee421f83ddf9c355e545c4', 'request_id': 'req-75f3508a-a73e-4c14-9edd-25ad2b4275fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.775395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29dba7f4f5ee421f83ddf9c355e545c4', 'request_id': 'req-75f3508a-a73e-4c14-9edd-25ad2b4275fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.789019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] The token cache is being invalidate because project 29dba7f4f5ee421f83ddf9c355e545c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:09.789385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.790044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.792343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1122/2209] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:09 2022] DELETE /identity/v3/projects/29dba7f4f5ee421f83ddf9c355e545c4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:09.796118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:09.807802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac33505-e3d5-4bed-9c1d-750fe5806b65 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:09.872434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:09.873316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:09.873594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:09.873867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] PATH_INFO: `/v3/users/26fe6ad7191047c68abee5322ca0e58b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:09.875052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26fe6ad7191047c68abee5322ca0e58b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:09.877159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:09.934881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26fe6ad7191047c68abee5322ca0e58b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.935548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26fe6ad7191047c68abee5322ca0e58b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.936259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26fe6ad7191047c68abee5322ca0e58b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.943494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26fe6ad7191047c68abee5322ca0e58b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.956099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:09.959155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8df72609-ac7e-4f75-83b7-67629437fb0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26fe6ad7191047c68abee5322ca0e58b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:09.975045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1088/2210] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:09 2022] DELETE /identity/v3/users/26fe6ad7191047c68abee5322ca0e58b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:10.253812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.255260 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:10.270716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3508a-a73e-4c14-9edd-25ad2b4275fc admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.334453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.335326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.335597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.335906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] PATH_INFO: `/v3/projects/a188de6d192d4ed79e9bbe32dbac455d` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.337286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a188de6d192d4ed79e9bbe32dbac455d)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.339673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.388017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a188de6d192d4ed79e9bbe32dbac455d', 'request_id': 'req-5c68b8ce-076b-40c9-a513-0760ec3572fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:10.394404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a188de6d192d4ed79e9bbe32dbac455d', 'request_id': 'req-5c68b8ce-076b-40c9-a513-0760ec3572fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:10.402521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a188de6d192d4ed79e9bbe32dbac455d', 'request_id': 'req-5c68b8ce-076b-40c9-a513-0760ec3572fc'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:10.414056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] The token cache is being invalidate because project a188de6d192d4ed79e9bbe32dbac455d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:10.414340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:10.415085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5c68b8ce-076b-40c9-a513-0760ec3572fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:10.416855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1123/2211] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:10 2022] DELETE /identity/v3/projects/a188de6d192d4ed79e9bbe32dbac455d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:10.421583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.421816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.422095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.440569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:10.442875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:10.505276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1089/2212] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:10.513806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.514096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.514373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.532669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:10.533886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:10.542480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1124/2213] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:10.546853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.556683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d0f4b-299d-49ed-8f1d-6881b4848278 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.561978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.562643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.562924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.563208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.563928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:10.564232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.565871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.577383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1090/2214] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:10.581520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.590695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ab0d38-193f-4210-ae83-0c76f8a4af10 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.595521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.596172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.596490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.596788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.597483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.599422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.625982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1125/2215] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:10 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:10.630106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.640875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69434c-dc15-4625-886f-7f10444aaee9 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.705591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.706262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.706541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.706816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.707513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.709559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.730809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1091/2216] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:10 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:10.735090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.746372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f690922-c4ff-4aa7-852b-2a0732f95d38 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.755050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.755050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.755050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.755050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.755469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.756062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.759057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:10.775661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.780365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1126/2217] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:21:10.786747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dce55ef-5d85-451c-8dc2-006d184e5473 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.791815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.792533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.792815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.793106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.793831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.795551 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32758-3634-4a4a-9380-27b2109c5801 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.812325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1092/2218] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:21:10.818690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:10.849667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7906e2-1a12-423a-ada8-d3df5fb65d66 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:10.856962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:10.858445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.859200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.859995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] PATH_INFO: `/v3/projects/0d708bd3eec34bf29000f10743da76ac/users/2cae6d642ae34f17a063217f3d6ecc98/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.886406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d708bd3eec34bf29000f10743da76ac, user_id=2cae6d642ae34f17a063217f3d6ecc98, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:10.889749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:10.903996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1127/2219] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:10 2022] PUT /identity/v3/projects/0d708bd3eec34bf29000f10743da76ac/users/2cae6d642ae34f17a063217f3d6ecc98/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:10.914625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:10.914921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:10.915202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:10.946266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] MFA Rules not processed for user `2cae6d642ae34f17a063217f3d6ecc98`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:10.947505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:11.007143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:11.046983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1093/2220] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:11.107195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:11.112732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:11.189879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce01c5-4706-4d72-9245-c2cc8ec08787 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:11.194663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:11.195549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:11.195840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:11.196125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:11.205446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:11.207513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97697f5e-cbaf-427f-8a05-f60e97bcda2c service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:11.217669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1128/2221] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:19.016157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.017981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.086329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f519cb-fb20-45da-b14d-e74929a81060 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:19.091416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:19.092324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.092598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.092873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] PATH_INFO: `/v3/users/7d643d155d8f44179c0b5410ddde038d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.094062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d643d155d8f44179c0b5410ddde038d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:19.096003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:19.149228 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d643d155d8f44179c0b5410ddde038d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.149903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d643d155d8f44179c0b5410ddde038d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.150529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d643d155d8f44179c0b5410ddde038d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.157336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d643d155d8f44179c0b5410ddde038d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.174509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d643d155d8f44179c0b5410ddde038d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.190357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1094/2222] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:19 2022] DELETE /identity/v3/users/7d643d155d8f44179c0b5410ddde038d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:19.195347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.196183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.196183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.215134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:19.216380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.283324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1129/2223] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:19.293023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.294519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.352261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bfc4a-cf4a-4e17-be78-fa2d92d0c3a4 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:19.356079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:19.356759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.357039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.357314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.366719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:19.368212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99e124-8745-4f4b-be26-f704cf1eb485 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:19.378414 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1095/2224] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:19.468389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.469852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.492326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6285950-c72a-4397-9608-997c62c28d98 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:19.497323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:19.498464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.498822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.499167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] PATH_INFO: `/v3/users/2cae6d642ae34f17a063217f3d6ecc98` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.506388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cae6d642ae34f17a063217f3d6ecc98)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:19.508956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:19.566207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cae6d642ae34f17a063217f3d6ecc98'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.566902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cae6d642ae34f17a063217f3d6ecc98'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.567549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cae6d642ae34f17a063217f3d6ecc98'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.575790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cae6d642ae34f17a063217f3d6ecc98'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.582786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cae6d642ae34f17a063217f3d6ecc98'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.590089 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:19.613558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1130/2225] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:19 2022] DELETE /identity/v3/users/2cae6d642ae34f17a063217f3d6ecc98 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:19.617875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.618257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.618587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.628065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:19.640252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:19.642267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.678917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.680777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.766861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1096/2226] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:19.768939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.770885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:19.800442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b486038-b170-4592-b41d-10ceac505b5e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:19.806484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:19.808044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.808139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.808679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] PATH_INFO: `/v3/projects/09a8ff92fd184f9e988166698e16e034` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.810050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a8ff92fd184f9e988166698e16e034)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:19.812957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:19.845108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a3d58-30f1-414a-b85f-d744ca227d97 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:19.849354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:19.850035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:19.850347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:19.852483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:19.863191 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:19.864723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:19.877358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1097/2227] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:19.881459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a8ff92fd184f9e988166698e16e034', 'request_id': 'req-c9098a01-a893-4257-8632-e366073e793f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.889359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a8ff92fd184f9e988166698e16e034', 'request_id': 'req-c9098a01-a893-4257-8632-e366073e793f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.900880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a8ff92fd184f9e988166698e16e034', 'request_id': 'req-c9098a01-a893-4257-8632-e366073e793f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.926708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] The token cache is being invalidate because project 09a8ff92fd184f9e988166698e16e034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:19.927956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.929677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:19.933598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1131/2228] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:19 2022] DELETE /identity/v3/projects/09a8ff92fd184f9e988166698e16e034 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:19.938881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:19.951439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8ade6-2863-466c-a936-cdf7d2eb698d service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.111039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.112045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.112432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.112774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] PATH_INFO: `/v3/users/7d33b112503c41ec97d9ebc8f2ee1ac6` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.114139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d33b112503c41ec97d9ebc8f2ee1ac6)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:20.116369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:20.198020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.199843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.209014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d33b112503c41ec97d9ebc8f2ee1ac6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.209691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d33b112503c41ec97d9ebc8f2ee1ac6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.210692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d33b112503c41ec97d9ebc8f2ee1ac6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.213579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:20.214667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9098a01-a893-4257-8632-e366073e793f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.219707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.220800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d33b112503c41ec97d9ebc8f2ee1ac6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.220922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.221105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.221382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] PATH_INFO: `/v3/users/b9b3f3fbf046451a9ec031844f8c35f6` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.222887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b3f3fbf046451a9ec031844f8c35f6)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:20.225772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:20.236325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d33b112503c41ec97d9ebc8f2ee1ac6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.267115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1098/2229] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:19 2022] DELETE /identity/v3/users/7d33b112503c41ec97d9ebc8f2ee1ac6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:20.290648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.298234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.329002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b3f3fbf046451a9ec031844f8c35f6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.329814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b3f3fbf046451a9ec031844f8c35f6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.330490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b3f3fbf046451a9ec031844f8c35f6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.340085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b3f3fbf046451a9ec031844f8c35f6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.349680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d39c22c0-1fb0-4d5e-9998-2fe0526597e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b3f3fbf046451a9ec031844f8c35f6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.371755 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:20.388796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1132/2230] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:20 2022] DELETE /identity/v3/users/b9b3f3fbf046451a9ec031844f8c35f6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:20.394865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.395210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.396382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.420048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce4c02-70bc-4cb6-8997-ccb619af3cb8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.423832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:20.425304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.513175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.514188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.514525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.515277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] PATH_INFO: `/v3/projects/0d708bd3eec34bf29000f10743da76ac` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.516202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d708bd3eec34bf29000f10743da76ac)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:20.518528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:20.569859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1133/2231] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:20.580258 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.582942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.597700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d708bd3eec34bf29000f10743da76ac', 'request_id': 'req-f28ebe62-b482-4688-b4e1-4a18008d3efd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.606983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d708bd3eec34bf29000f10743da76ac', 'request_id': 'req-f28ebe62-b482-4688-b4e1-4a18008d3efd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.617969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d708bd3eec34bf29000f10743da76ac', 'request_id': 'req-f28ebe62-b482-4688-b4e1-4a18008d3efd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.631915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] The token cache is being invalidate because project 0d708bd3eec34bf29000f10743da76ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:20.632209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.632924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f28ebe62-b482-4688-b4e1-4a18008d3efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.634653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1099/2232] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:20 2022] DELETE /identity/v3/projects/0d708bd3eec34bf29000f10743da76ac => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:20.641479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.642040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.642837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.656971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:20.668899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61deca4-c672-4f57-bde8-eb99d45ea4fb None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.671138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:20.672667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.725709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.726299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.726581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.726897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.729754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.744810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1100/2233] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:20.745605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.746868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.759017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f05a3-4c4b-4233-a8ae-6c0518aeaea9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.763790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.764671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.764954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.765220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] PATH_INFO: `/v3/projects/f7985cfd55bf43ce9038a87d9074209c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.766410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7985cfd55bf43ce9038a87d9074209c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:20.768324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:20.810807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:20.812272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a73253-1e86-482e-ad2d-5919a8ffe5ac service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:20.816754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:20.825395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1134/2234] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:20.827034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:20.827364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:20.827678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:20.850434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7985cfd55bf43ce9038a87d9074209c', 'request_id': 'req-3f991c60-1509-4702-a59e-0c15ff0d3c7b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.852993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7985cfd55bf43ce9038a87d9074209c', 'request_id': 'req-3f991c60-1509-4702-a59e-0c15ff0d3c7b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.855974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:20.857568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.866341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7985cfd55bf43ce9038a87d9074209c', 'request_id': 'req-3f991c60-1509-4702-a59e-0c15ff0d3c7b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.901536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] The token cache is being invalidate because project f7985cfd55bf43ce9038a87d9074209c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:20.901969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.904131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:20.906725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1101/2235] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:20 2022] DELETE /identity/v3/projects/f7985cfd55bf43ce9038a87d9074209c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:20.912939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.914609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:20.960187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1135/2236] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:20.964855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:20.981275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855a9b43-1601-4e4f-bdb2-dc379d01d00f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:20.996700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:20.999901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.000213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.000644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.002749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:21.003062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.006039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.015671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f991c60-1509-4702-a59e-0c15ff0d3c7b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.021975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.023106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.023455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.023878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] PATH_INFO: `/v3/users/c86df20b1502499d905803f5b491013b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.025207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86df20b1502499d905803f5b491013b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.028457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.052711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1136/2237] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:21.054554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.065852 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4464c3-56c6-4096-96ca-d458a923ff34 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.073978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.075702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.076569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.076978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.077952 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.080343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.106644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:21.117531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1137/2238] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:21.118599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.131350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86df20b1502499d905803f5b491013b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.132369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86df20b1502499d905803f5b491013b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.133481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86df20b1502499d905803f5b491013b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.165143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86df20b1502499d905803f5b491013b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.182317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86df20b1502499d905803f5b491013b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.202001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1102/2239] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:20 2022] DELETE /identity/v3/users/c86df20b1502499d905803f5b491013b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:21.203174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946ba02-389c-4e7b-bf1e-96864b051366 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.256837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.258325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.269818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.270506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.270790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.271533 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.272486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.275025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.307768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:21.315349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1138/2240] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:21.321474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.324582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.343896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33151d67-d84b-4d44-8e45-c104a387d452 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.351131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.351905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.352508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.352508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.354767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.356695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.367287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ffdf9-4abc-457e-973f-5cc965665c61 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.372358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1139/2241] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:21.374243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.374386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.375296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.375692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.376068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] PATH_INFO: `/v3/projects/8e9c8b24824e4336b52f0ce5dfd84d81` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.377495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e9c8b24824e4336b52f0ce5dfd84d81)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.379847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.388161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866afe1f-0a64-4e76-8f23-5cba41cfa704 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.393863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.395498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.395939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.396396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.397652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.400036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.415724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1140/2242] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:21.417559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.431068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76987f4c-a74a-4460-816d-f73416a60769 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.438216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.439505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.439902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.440315 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] PATH_INFO: `/v3/projects/cb659dae6f4b4f26a2d869138f573561/users/927951dcdbb348d6ac8d467753c55d61/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.452512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e9c8b24824e4336b52f0ce5dfd84d81', 'request_id': 'req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.458724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb659dae6f4b4f26a2d869138f573561, user_id=927951dcdbb348d6ac8d467753c55d61, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.460323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e9c8b24824e4336b52f0ce5dfd84d81', 'request_id': 'req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.461660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa796f-4063-41ac-b8fc-5fd306741441 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.471043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e9c8b24824e4336b52f0ce5dfd84d81', 'request_id': 'req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.483097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1141/2243] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:21 2022] PUT /identity/v3/projects/cb659dae6f4b4f26a2d869138f573561/users/927951dcdbb348d6ac8d467753c55d61/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:21.488332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.488710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.489057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.490007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] The token cache is being invalidate because project 8e9c8b24824e4336b52f0ce5dfd84d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:21.490302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.490980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.493144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1103/2244] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:21 2022] DELETE /identity/v3/projects/8e9c8b24824e4336b52f0ce5dfd84d81 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:21.497205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.499038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.507686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:21.508927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.567526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:21.582718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1142/2245] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:21.585978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.585978 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.605964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28987fc7-7259-4337-bd84-c9f01defc21d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.611284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.612421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.612638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.612897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] PATH_INFO: `/v3/projects/2ed3c09a34a04ebe826fc93487a307cf` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.614119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed3c09a34a04ebe826fc93487a307cf)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.616347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.616468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6cffd-6e0b-4379-bcd8-71ce1906f6a2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.622092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.623075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.623430 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.623789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] PATH_INFO: `/v3/users/b7dd288c0b974125b98b772f253aa3db` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.625274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7dd288c0b974125b98b772f253aa3db)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.627285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.678795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed3c09a34a04ebe826fc93487a307cf', 'request_id': 'req-e358084a-b042-4038-a374-e0b9914226f9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.685511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed3c09a34a04ebe826fc93487a307cf', 'request_id': 'req-e358084a-b042-4038-a374-e0b9914226f9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.692695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7dd288c0b974125b98b772f253aa3db'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.693461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7dd288c0b974125b98b772f253aa3db'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.693604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed3c09a34a04ebe826fc93487a307cf', 'request_id': 'req-e358084a-b042-4038-a374-e0b9914226f9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.694096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7dd288c0b974125b98b772f253aa3db'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.704400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7dd288c0b974125b98b772f253aa3db'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.707152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] The token cache is being invalidate because project 2ed3c09a34a04ebe826fc93487a307cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:21.707418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.708108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.709733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1143/2246] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:21 2022] DELETE /identity/v3/projects/2ed3c09a34a04ebe826fc93487a307cf => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:21.713275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.713275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.729941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-4b31ac8a-b68f-40db-902c-a59af5e1b884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7dd288c0b974125b98b772f253aa3db'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:21.748572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1104/2247] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:21 2022] DELETE /identity/v3/users/b7dd288c0b974125b98b772f253aa3db => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:21.752987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49a8c-96ad-439e-b469-46649d7712eb None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.753616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49a8c-96ad-439e-b469-46649d7712eb None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.754052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49a8c-96ad-439e-b469-46649d7712eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.770545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:21.792171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f6e49a8c-96ad-439e-b469-46649d7712eb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:21.792709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358084a-b042-4038-a374-e0b9914226f9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.793646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f6e49a8c-96ad-439e-b469-46649d7712eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.854658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.855467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.855990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.856276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.858369 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.868499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1105/2248] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:21.871607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.871998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.872421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.876963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.878412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.889302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1144/2249] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:21.893742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:21.895216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:21.904873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1106/2250] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:21.909288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.919653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c68d3-b3e8-4258-b8a7-d5b7a80ba40f service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.925331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.926206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.926559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.926922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.927721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:21.928256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.930072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:21.944434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1145/2251] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:21.949490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:21.960492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a4b51-eb8b-473e-be72-7602969e8381 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:21.967240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:21.968046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:21.968539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:21.968819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:21.969559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:21.971948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.019535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1107/2252] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:21 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:22.020905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.033706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1a5b2-c04a-41ae-9621-f08478a8517e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.151308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.152012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.152335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.152609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.153648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.155259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.180211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1146/2253] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:22 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:21:22.184719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.196983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78190-5dfe-48a5-b2c1-b2b6f113ef7c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.202940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.203827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.204206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.204581 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.205408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.207226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.219984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1108/2254] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:21:22.224103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.235003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea65d4f-ac9a-4f99-afe0-d0fec2338f7c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.240599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.241274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.241562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.241840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.242578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.244843 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41124840-305e-4ed3-ac9a-2e5b6306acbb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.256930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1147/2255] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:22.261742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.273145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a5336-4df2-4eb8-acaf-2d0fee00e67a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.281983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.283570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.284150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.284577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] PATH_INFO: `/v3/projects/90f5e90815264337832c864f128c13f3/users/de772bdc29e6497684982986662e42c7/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.302285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f5e90815264337832c864f128c13f3, user_id=de772bdc29e6497684982986662e42c7, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.305393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.308779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:22.326152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1109/2256] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:22 2022] PUT /identity/v3/projects/90f5e90815264337832c864f128c13f3/users/de772bdc29e6497684982986662e42c7/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:21:22.337285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.337740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.338177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.378311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] MFA Rules not processed for user `de772bdc29e6497684982986662e42c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:22.380787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.405266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.407970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.488991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:22.518992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1148/2257] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:22.528703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.531282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.573212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265a9b7-38d2-45a8-8773-b28aa2449342 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.578974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.579652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.580047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.580431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] PATH_INFO: `/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.581957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40dc3201d4ef40c0a836c9b7d620b0c3)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.586335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.628655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42560e20-e12b-4edf-8964-7e587c6c7b22 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.641485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40dc3201d4ef40c0a836c9b7d620b0c3', 'request_id': 'req-02abb0c0-81d1-4da4-a281-a995605dc0f7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:22.649379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40dc3201d4ef40c0a836c9b7d620b0c3', 'request_id': 'req-02abb0c0-81d1-4da4-a281-a995605dc0f7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:22.658532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40dc3201d4ef40c0a836c9b7d620b0c3', 'request_id': 'req-02abb0c0-81d1-4da4-a281-a995605dc0f7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:22.678506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] The token cache is being invalidate because project 40dc3201d4ef40c0a836c9b7d620b0c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:22.679046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:22.679488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02abb0c0-81d1-4da4-a281-a995605dc0f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:22.683535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1110/2258] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:22 2022] DELETE /identity/v3/projects/40dc3201d4ef40c0a836c9b7d620b0c3 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:22.688223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.688868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.689148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.700154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.700920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.701267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.701607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.703511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.708239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:22.709636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.802131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.803867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08023143-12d9-4a9e-aff5-f02f72a07ef0 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.819091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1149/2259] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:22.882166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1111/2260] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:22.890924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.891261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.891678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.910730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:22.912251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:22.923969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1150/2261] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:22.928905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.942654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89baaa1-3f65-4b3a-b8be-6140c580c8e7 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.948506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.949253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.949646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.949979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.950797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:22.951169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.953028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:22.966978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1112/2262] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:22.971741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:22.982485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e90ca-e397-41be-b101-e3dd73afb154 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:22.989210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:22.990128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:22.990429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:22.990807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:22.991572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:22.993768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.037900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.048220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1151/2263] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:22 2022] POST /identity/v3/projects => generated 367 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:23.052945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216f22f-ef6f-4b0f-8105-b774bb4cb939 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:23.071016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:23.178873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:23.179567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.179845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.180114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.180900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:23.189036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.225974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1113/2264] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:23 2022] POST /identity/v3/users => generated 284 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:23.230902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.257601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8efd46-ae82-446d-9fb6-ed0b22ac6565 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:23.264974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:23.265837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.266188 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.266806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.268109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:23.270347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.287354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1152/2265] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:23.292367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.314518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584568a5-b1cf-4623-91d7-0bc684f6d7c6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:23.329314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:23.330494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.330941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.331285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.331976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:23.333804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccaf3c-ce2d-4a88-a1e5-b8dc4fe6d341 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.366252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1114/2266] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:23.367720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.382963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954164c0-bb66-4b4f-acc8-4ffe933534a5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:23.399080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:23.400107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.400485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.400777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] PATH_INFO: `/v3/projects/32cefea3e0f441468aab2f8799b70873/users/2a6fe98e037a46b2aba5e9b9391a1142/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.430054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=32cefea3e0f441468aab2f8799b70873, user_id=2a6fe98e037a46b2aba5e9b9391a1142, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:23.433539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.447025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:23.455122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1153/2267] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:23 2022] PUT /identity/v3/projects/32cefea3e0f441468aab2f8799b70873/users/2a6fe98e037a46b2aba5e9b9391a1142/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:23.467605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.467946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.469723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.504099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] MFA Rules not processed for user `2a6fe98e037a46b2aba5e9b9391a1142`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:23.505345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:23.671518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1115/2268] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:23.673046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.831377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d01a81-dea3-47d6-92dd-48f208550e1f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:23.844475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:23.845170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:23.845454 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:23.845795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:23.846566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:23.848552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:23.898054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1154/2269] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:23 2022] POST /identity/v3/projects => generated 367 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:23.903353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:23.916827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903b416-1913-4b51-b917-c0aaa18d7808 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.004601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.005420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.005892 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.006236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.006957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.008983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.043667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1116/2270] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:23 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:21:24.047741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.078070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dac88f6-8cc6-42ea-a406-d112dcec9470 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.097224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.101721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.102329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.103130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.105835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.109090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.135742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1155/2271] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:24.139895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.163770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196dcc6-f7a5-4961-85fc-e942b5bfae9c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.199583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.201063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.201531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.202634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] PATH_INFO: `/v3/projects/18eb62750d854fdeb8b17da881b4f5ce/users/be35715cc93f4718a26d6ff0e2d26d66/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.208095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:24.234836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eb62750d854fdeb8b17da881b4f5ce, user_id=be35715cc93f4718a26d6ff0e2d26d66, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.240167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.262017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1117/2272] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:24 2022] PUT /identity/v3/projects/18eb62750d854fdeb8b17da881b4f5ce/users/be35715cc93f4718a26d6ff0e2d26d66/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:24.266129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.277727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f061be8f-ddb9-4f87-9605-cf687ba3846b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.349705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.350470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.350670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.350968 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.351693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.353651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d990217-11d1-470e-8b1d-cd2687cac60a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.364604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1156/2273] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:24.368721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.380763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5a222a-ff37-495a-b4b4-91c4d5722085 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.387777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.389002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.389373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.390008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] PATH_INFO: `/v3/projects/18eb62750d854fdeb8b17da881b4f5ce/users/be35715cc93f4718a26d6ff0e2d26d66/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.391889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=18eb62750d854fdeb8b17da881b4f5ce, user_id=be35715cc93f4718a26d6ff0e2d26d66, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.396173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.410798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1118/2274] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:24 2022] PUT /identity/v3/projects/18eb62750d854fdeb8b17da881b4f5ce/users/be35715cc93f4718a26d6ff0e2d26d66/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:24.421525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117abaee-9271-402c-9d89-837604df9bae None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.422095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117abaee-9271-402c-9d89-837604df9bae None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.422650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117abaee-9271-402c-9d89-837604df9bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.459883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-117abaee-9271-402c-9d89-837604df9bae None None] MFA Rules not processed for user `be35715cc93f4718a26d6ff0e2d26d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:24.465598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-117abaee-9271-402c-9d89-837604df9bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:24.571711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:24.592435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1157/2275] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:24.605458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.607361 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:24.688989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea99b40-079f-4cc0-be99-31056ab1a8ca admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.693261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.693994 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.694353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.694727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:24.711607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:24.713817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218c3ff-07f0-4377-9f81-f8617266be54 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:24.726102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1119/2276] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:24 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:24.876956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117abaee-9271-402c-9d89-837604df9bae None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:24.879232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-117abaee-9271-402c-9d89-837604df9bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:24.907328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117abaee-9271-402c-9d89-837604df9bae None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:24.922087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:24.932336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:24.932654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:24.932936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:25.022298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:25.024440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:25.037245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1158/2277] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:24 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:27.701648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:27.702174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:27.702402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:27.739020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] MFA Rules not processed for user `927951dcdbb348d6ac8d467753c55d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:27.740266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:27.816005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1120/2278] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:27.820706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:27.822525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:27.921529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bc7b6-30ac-420e-850f-908bbfcb9f76 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:27.926106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:27.926772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:27.927054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:27.927329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:27.928057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:27.930000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:27.960187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1159/2279] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:27 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:27.961410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:27.972324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e5f41-03b6-4a5b-b95b-37a316127b3d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:28.033863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:28.034560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:28.034937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:28.035213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:28.035916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:28.037902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:28.047158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:28.061744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1121/2280] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:27 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:28.065934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:28.084686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98f9ac-957f-47b7-b9f9-afd7cd4ea4c6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:28.090150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:28.090824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:28.091214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:28.091499 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:28.092239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:28.093871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:28.105803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1160/2281] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:28.109750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:28.120025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ee38b-c56d-4af0-a86f-a1755ff5ad9a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:28.124815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:28.125690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:28.125968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:28.126243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] PATH_INFO: `/v3/projects/b44bf3050ca24ccaa09e4620be92f794/users/7fc22e84cc7c480180e553ff122e7722/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:28.142998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44bf3050ca24ccaa09e4620be92f794, user_id=7fc22e84cc7c480180e553ff122e7722, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:28.145800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:28.161576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1122/2282] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:28 2022] PUT /identity/v3/projects/b44bf3050ca24ccaa09e4620be92f794/users/7fc22e84cc7c480180e553ff122e7722/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:28.165610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:28.188108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3b7ef-4b0c-4a6d-ab69-f2e6abd8a52a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:28.266327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:28.267001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:28.267282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:28.267554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:28.268294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:28.269906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:28.283662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1161/2283] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:28.284803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:28.298456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d1566a-7bcb-41c4-8051-3f6912e6ac16 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:28.304115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:28.305404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:28.305864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:28.306325 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] PATH_INFO: `/v3/projects/b44bf3050ca24ccaa09e4620be92f794/users/7fc22e84cc7c480180e553ff122e7722/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:28.308605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44bf3050ca24ccaa09e4620be92f794, user_id=7fc22e84cc7c480180e553ff122e7722, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:28.311301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f2d3a-01dc-4c4d-9578-305b520edb4f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:28.331684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1123/2284] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:28 2022] PUT /identity/v3/projects/b44bf3050ca24ccaa09e4620be92f794/users/7fc22e84cc7c480180e553ff122e7722/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:28.960881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:28.968806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:28.977183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbbbdd-3471-413d-9ca9-5318dd4508e7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:29.086583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:29.087494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:29.088113 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:29.088571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] PATH_INFO: `/v3/users/2a6fe98e037a46b2aba5e9b9391a1142` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:29.089794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a6fe98e037a46b2aba5e9b9391a1142)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:29.092068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:29.160920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a6fe98e037a46b2aba5e9b9391a1142'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.162560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a6fe98e037a46b2aba5e9b9391a1142'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.163447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a6fe98e037a46b2aba5e9b9391a1142'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.175745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a6fe98e037a46b2aba5e9b9391a1142'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.188511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a6fe98e037a46b2aba5e9b9391a1142'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.220621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1162/2285] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:28 2022] DELETE /identity/v3/users/2a6fe98e037a46b2aba5e9b9391a1142 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:29.222123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:29.222583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:29.222660 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:29.252549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:29.253666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:29.322910 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1124/2286] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:21:29.333998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:29.338080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:29.409783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab88830-2a18-45a1-ab4c-a1ad7db324b7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:29.413922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:29.414650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:29.415330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:29.415330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:29.429644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:29.431364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:29.482220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1163/2287] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:29.794518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:29.796754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:29.822854 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83a86-9a3a-4ce3-8ed2-db47bfcf3b2e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:29.827694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:29.829914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:29.830402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:29.830868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] PATH_INFO: `/v3/projects/32cefea3e0f441468aab2f8799b70873` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:29.832905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32cefea3e0f441468aab2f8799b70873)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:29.837318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:29.911887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32cefea3e0f441468aab2f8799b70873', 'request_id': 'req-07242c70-78fe-4a47-9106-d47010469be9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.918470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32cefea3e0f441468aab2f8799b70873', 'request_id': 'req-07242c70-78fe-4a47-9106-d47010469be9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.929285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32cefea3e0f441468aab2f8799b70873', 'request_id': 'req-07242c70-78fe-4a47-9106-d47010469be9'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.954252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] The token cache is being invalidate because project 32cefea3e0f441468aab2f8799b70873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:29.954548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.955267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:29.965460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:29.972349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1125/2288] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:29 2022] DELETE /identity/v3/projects/32cefea3e0f441468aab2f8799b70873 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:29.977135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd92f14-3e4b-49fc-b896-0cf696d7734b service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:30.029422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:30.131326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:30.132595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:30.133107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:30.133581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] PATH_INFO: `/v3/users/be35715cc93f4718a26d6ff0e2d26d66` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:30.135779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be35715cc93f4718a26d6ff0e2d26d66)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:30.137924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:30.212155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be35715cc93f4718a26d6ff0e2d26d66'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:30.213649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be35715cc93f4718a26d6ff0e2d26d66'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:30.214314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be35715cc93f4718a26d6ff0e2d26d66'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:30.224638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be35715cc93f4718a26d6ff0e2d26d66'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:30.244900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5bee1540-9c2c-4b26-b3a8-69fa2277e328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be35715cc93f4718a26d6ff0e2d26d66'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:30.276451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1164/2289] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:21:29 2022] DELETE /identity/v3/users/be35715cc93f4718a26d6ff0e2d26d66 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:30.788891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:30.791823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:30.817201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07242c70-78fe-4a47-9106-d47010469be9 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:30.896555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:30.939784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:30.941606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:30.942567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:30.942941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] PATH_INFO: `/v3/projects/18eb62750d854fdeb8b17da881b4f5ce` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:30.944572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18eb62750d854fdeb8b17da881b4f5ce)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:30.946809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.092470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18eb62750d854fdeb8b17da881b4f5ce', 'request_id': 'req-35682de8-227a-43f0-bc04-1601882c75f6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:31.102708 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18eb62750d854fdeb8b17da881b4f5ce', 'request_id': 'req-35682de8-227a-43f0-bc04-1601882c75f6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:31.113898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18eb62750d854fdeb8b17da881b4f5ce', 'request_id': 'req-35682de8-227a-43f0-bc04-1601882c75f6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:31.143072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] The token cache is being invalidate because project 18eb62750d854fdeb8b17da881b4f5ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:21:31.143296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:31.144323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35682de8-227a-43f0-bc04-1601882c75f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:21:31.147041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1126/2290] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:21:30 2022] DELETE /identity/v3/projects/18eb62750d854fdeb8b17da881b4f5ce => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:31.154194 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.154639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.154753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.195592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:31.198295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:31.436927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1165/2291] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:31.439817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.440111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.440587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.460912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:31.463310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:31.483230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1127/2292] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:31.487304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.500238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b92f7d-4400-4a8e-b957-fe8fed2d20cf None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.517273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.517879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.518168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.519037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.524597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:21:31.525041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:31.527282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.556451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1166/2293] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:21:31.559800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.574576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83223b79-34cd-4179-8b6d-3f0d83abfdfe None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.580897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.581577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.581863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.582143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.582844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:31.584992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.639418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1128/2294] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:21:31 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:31.640864 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.661931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368949-c656-4f24-8303-56e0965d7c34 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.706236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:31.760370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.761479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.761918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.762357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.763237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:31.765575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.803949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.808800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1167/2295] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:21:31 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:21:31.820485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1f6cd4-e4d4-4c51-9d36-60ede5a3840d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.827553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.828250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.828740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.829024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.829771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:31.831762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.850290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1129/2296] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:31.851323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.870219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5907-ca17-4f33-bb72-2b8fb71c25b2 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.897426 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.898442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.898961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.899196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:31.899969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:31.901795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e68f8f8-eeaa-4ed1-a828-d3b2d9683773 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:31.927300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1168/2297] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:21:31.931844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:31.962027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e4133-8f80-4760-b0bd-2e93f8dc30b9 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:31.985595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:31.986524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:31.986804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:31.987077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] PATH_INFO: `/v3/projects/f7c8c75b264644b7af9831e14cbf7841/users/2b27ee7d15ca42bca6a09dd18751f01e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:32.038323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c8c75b264644b7af9831e14cbf7841, user_id=2b27ee7d15ca42bca6a09dd18751f01e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:32.041345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:32.065638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1130/2298] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:21:31 2022] PUT /identity/v3/projects/f7c8c75b264644b7af9831e14cbf7841/users/2b27ee7d15ca42bca6a09dd18751f01e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:21:32.072793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5716c-3bf8-41d5-8720-e9c22c8ca2c6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:32.073137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5716c-3bf8-41d5-8720-e9c22c8ca2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:32.073362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5716c-3bf8-41d5-8720-e9c22c8ca2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:32.113786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b6b5716c-3bf8-41d5-8720-e9c22c8ca2c6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:32.115035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b6b5716c-3bf8-41d5-8720-e9c22c8ca2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:32.238218 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1169/2299] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:32.246776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:32.249143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:32.360593 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a1db-f7b4-4ed9-a909-8b97a3cd8d2a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:32.368147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:32.369633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:32.370029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:32.370403 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:32.385321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:32.387926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:32.409233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1131/2300] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:21:34.016200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d89da-41c8-4678-94b0-1128af000066 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:34.017214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d89da-41c8-4678-94b0-1128af000066 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:34.017682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d89da-41c8-4678-94b0-1128af000066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:34.054058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bb6d89da-41c8-4678-94b0-1128af000066 None None] MFA Rules not processed for user `2b27ee7d15ca42bca6a09dd18751f01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:34.055638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bb6d89da-41c8-4678-94b0-1128af000066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:34.177458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1170/2301] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:34.205465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:34.207488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:34.225920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:34.299370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85031fa9-ed30-4b93-b047-4c3a550daffe service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:34.306227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:34.307674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:34.308163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:34.308928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:34.359043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:34.367482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d13fd4-db02-4a23-8729-d171b8d1ed1b service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:34.401332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1132/2302] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:34.450268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:34.451151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:34.451483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:34.540411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] MFA Rules not processed for user `7fc22e84cc7c480180e553ff122e7722`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:34.541725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:34.586663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:34.692080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1171/2303] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:36.032463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243550a-ed0b-46e5-a552-e75af12c0306 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:36.033355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243550a-ed0b-46e5-a552-e75af12c0306 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:36.033817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243550a-ed0b-46e5-a552-e75af12c0306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:36.051533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-6243550a-ed0b-46e5-a552-e75af12c0306 None None] MFA Rules not processed for user `2b27ee7d15ca42bca6a09dd18751f01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:36.052901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6243550a-ed0b-46e5-a552-e75af12c0306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:36.060888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1133/2304] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:41.864316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:41.865649 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:41.925169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f8dfd-e793-4c0f-8726-b097c298d45d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:41.928690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:41.929432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:41.929711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:41.929984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:41.931739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:41.991419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:41.992923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:42.001945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1172/2305] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:21:42.160176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b2c-f3a2-4d6f-9aac-d1046afe6c05 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:42.160652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b2c-f3a2-4d6f-9aac-d1046afe6c05 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:42.160752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649b2c-f3a2-4d6f-9aac-d1046afe6c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:42.179633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5b649b2c-f3a2-4d6f-9aac-d1046afe6c05 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:42.180840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5b649b2c-f3a2-4d6f-9aac-d1046afe6c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:42.248100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1134/2306] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:21:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:42.293476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:42.295144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:42.365685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5773b4f-b57b-47f8-991d-28c52a099c81 service glance] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:42.370750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:42.371913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:42.372242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:42.372557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:42.380899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:42.382579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4ddcd-01f9-4cae-befd-257471a92114 service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:42.391338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1173/2307] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:21:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:21:51.576080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:51.576691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:51.576691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:51.608221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] MFA Rules not processed for user `927951dcdbb348d6ac8d467753c55d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:51.609946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:51.650256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1135/2308] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:21:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:52.387986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:21:55.911270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1bd30-6991-4cb9-a4fb-c3c8464db479 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:55.911899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1bd30-6991-4cb9-a4fb-c3c8464db479 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:55.911899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1bd30-6991-4cb9-a4fb-c3c8464db479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:55.936703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-00c1bd30-6991-4cb9-a4fb-c3c8464db479 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:21:55.938329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-00c1bd30-6991-4cb9-a4fb-c3c8464db479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:21:55.971298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1174/2309] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:21:55.984519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:21:56.003539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817e87da-213e-40d1-bf78-10638e49ca60 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:21:56.008970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:21:56.009632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:21:56.009914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:21:56.010229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:21:56.020916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:21:56.022561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:21:56.034353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1136/2310] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:21:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:22:05.958059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:06.029840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=74 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:06.384515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e68d87-0417-444e-bc61-82980e40c55f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:06.385047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e68d87-0417-444e-bc61-82980e40c55f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:06.386206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e68d87-0417-444e-bc61-82980e40c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:06.408785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a0e68d87-0417-444e-bc61-82980e40c55f None None] MFA Rules not processed for user `2d8a7414fd254ec981d35b417dc373cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:06.411125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a0e68d87-0417-444e-bc61-82980e40c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:06.495355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1175/2311] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:22:12.072418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:12.073812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:12.119575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb81541-21ad-4a91-b0c2-0320e34d17d9 service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:12.124029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:12.124921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:12.125202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:12.125476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] PATH_INFO: `/v3/users/c4dd4a24951b43d7a2e46af5c73d5486` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:12.126676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4dd4a24951b43d7a2e46af5c73d5486)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:12.128651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:12.175434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4dd4a24951b43d7a2e46af5c73d5486'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:12.176068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4dd4a24951b43d7a2e46af5c73d5486'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:12.176678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4dd4a24951b43d7a2e46af5c73d5486'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:12.183370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4dd4a24951b43d7a2e46af5c73d5486'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:12.194795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4dd4a24951b43d7a2e46af5c73d5486'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:12.208914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1137/2312] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:22:12 2022] DELETE /identity/v3/users/c4dd4a24951b43d7a2e46af5c73d5486 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:12.213580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:12.214023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:12.214690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:12.251444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:12.253152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:12.335366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1176/2313] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:12.351051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:12.352590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:12.463304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ccc75-9289-4c27-ab8a-c7bd7f9545cc admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:12.468525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:12.469339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:12.469627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:12.469957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:12.480335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:12.482385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:12.492869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1138/2314] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:12.991710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:12.993199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:13.006079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb58725e-4ae7-4a12-bdf3-9ab487df65b7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:13.010668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:13.011538 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:13.011819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:13.012099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] PATH_INFO: `/v3/projects/eae15379454d4207b73229abbc4d23ac` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:13.013947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eae15379454d4207b73229abbc4d23ac)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:13.015798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:13.080427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eae15379454d4207b73229abbc4d23ac', 'request_id': 'req-5b655b64-da01-42ad-8dd3-baef1a14d930'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:13.088655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eae15379454d4207b73229abbc4d23ac', 'request_id': 'req-5b655b64-da01-42ad-8dd3-baef1a14d930'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:13.096544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eae15379454d4207b73229abbc4d23ac', 'request_id': 'req-5b655b64-da01-42ad-8dd3-baef1a14d930'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:13.107595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] The token cache is being invalidate because project eae15379454d4207b73229abbc4d23ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:22:13.108133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:13.108789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5b655b64-da01-42ad-8dd3-baef1a14d930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:13.110326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1177/2315] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:22:12 2022] DELETE /identity/v3/projects/eae15379454d4207b73229abbc4d23ac => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:17.704051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:17.705465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:17.717882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145ad8a2-eca2-433c-bbd7-9866da742be7 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:17.742022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:17.775833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:17.776739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:17.777022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:17.777295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] PATH_INFO: `/v3/users/de772bdc29e6497684982986662e42c7` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:17.778525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de772bdc29e6497684982986662e42c7)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:17.780535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:17.827014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de772bdc29e6497684982986662e42c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:17.827652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de772bdc29e6497684982986662e42c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:17.828245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de772bdc29e6497684982986662e42c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:17.835215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de772bdc29e6497684982986662e42c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:17.847368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de772bdc29e6497684982986662e42c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:17.860537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1139/2316] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:22:17 2022] DELETE /identity/v3/users/de772bdc29e6497684982986662e42c7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:17.865067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:17.865618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:17.865715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:17.884138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:17.885376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:17.901615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:17.948167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1178/2317] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:17.957869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:17.960153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:18.067354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055976-208e-4f61-8999-34217750b5f5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:18.078126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:18.078820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:18.079097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:18.079373 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:18.100263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:18.112327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:18.140366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1140/2318] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:18.610739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:18.612895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:18.631513 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce865f0-9669-4b4a-a904-17a8e7f57d73 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:18.636228 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:18.637402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:18.637614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:18.637955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] PATH_INFO: `/v3/projects/90f5e90815264337832c864f128c13f3` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:18.639554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f5e90815264337832c864f128c13f3)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:18.641311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:18.703019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f5e90815264337832c864f128c13f3', 'request_id': 'req-e1aabf24-83e6-447f-a73a-25d10ddb9d17'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:18.709640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f5e90815264337832c864f128c13f3', 'request_id': 'req-e1aabf24-83e6-447f-a73a-25d10ddb9d17'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:18.719355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f5e90815264337832c864f128c13f3', 'request_id': 'req-e1aabf24-83e6-447f-a73a-25d10ddb9d17'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:18.735314 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] The token cache is being invalidate because project 90f5e90815264337832c864f128c13f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:22:18.735607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:18.736511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e1aabf24-83e6-447f-a73a-25d10ddb9d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:18.738966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1179/2319] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:22:18 2022] DELETE /identity/v3/projects/90f5e90815264337832c864f128c13f3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:22.778026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:22.779456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:22.792190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad41d4-9a4a-4096-9d19-506ad435bbab service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:22.836571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:22.849335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:22.850207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:22.850490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:22.850785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] PATH_INFO: `/v3/users/aac74f78cfa64c32be3f495125ff8d59` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:22.851946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aac74f78cfa64c32be3f495125ff8d59)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:22.854010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:22.900255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aac74f78cfa64c32be3f495125ff8d59'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:22.900927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aac74f78cfa64c32be3f495125ff8d59'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:22.901511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aac74f78cfa64c32be3f495125ff8d59'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:22.908521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aac74f78cfa64c32be3f495125ff8d59'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:22.920352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aac74f78cfa64c32be3f495125ff8d59'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:22.933255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1141/2320] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:22:22 2022] DELETE /identity/v3/users/aac74f78cfa64c32be3f495125ff8d59 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:22.938328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:22.938791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:22.939003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:22.958162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:22.959466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:23.020890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1180/2321] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:23.030315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.031841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:23.089912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ecda4-09ee-4320-a924-6417ca5c5dd0 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.094123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.094772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.095057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.095339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.104370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.106486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da868eed-5ae3-4362-ae77-edecaea646ff service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.116204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1142/2322] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:23.460125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.461749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:23.476583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43bd7-afa3-4921-84a3-30a3a0549a65 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.481993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.483124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.483408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.483789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] PATH_INFO: `/v3/projects/667d4eb3be234d729065069069b06fd4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.485083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] RBAC: Authorizing `identity:delete_project(project_id=667d4eb3be234d729065069069b06fd4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.487317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.528505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:23.536544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667d4eb3be234d729065069069b06fd4', 'request_id': 'req-a1afe125-de32-418d-9149-3f87b120909b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:23.541969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667d4eb3be234d729065069069b06fd4', 'request_id': 'req-a1afe125-de32-418d-9149-3f87b120909b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:23.549557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667d4eb3be234d729065069069b06fd4', 'request_id': 'req-a1afe125-de32-418d-9149-3f87b120909b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:23.560462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] The token cache is being invalidate because project 667d4eb3be234d729065069069b06fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:22:23.560743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:23.561351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a1afe125-de32-418d-9149-3f87b120909b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:23.562876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1181/2323] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:22:23 2022] DELETE /identity/v3/projects/667d4eb3be234d729065069069b06fd4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:23.567760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.568060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.568371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.616190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:23.617415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:23.657433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:23.679938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1143/2324] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:23.687897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.688183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.688489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.707946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:23.709161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:23.717998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1182/2325] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:23.723344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.735047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f3ed5-4fd9-41ba-9998-d6ede1b99725 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.740272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.740959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.741241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.741521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.742309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:22:23.743020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.744671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.759058 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1144/2326] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:22:23.763698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.773609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd258c5-e27f-414c-a363-b6792fea276f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.779158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.779866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.780219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.780659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.781447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.783525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.814665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1183/2327] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:22:23 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:22:23.818834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.837646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d934438-0fff-4f1b-b30e-a345a0335549 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.891838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.892543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.892834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.893114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.893815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.895766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.916035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1145/2328] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:22:23 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:22:23.920220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.930237 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3a31f-b1b5-4969-bc2e-5f125add3d00 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.934795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.935451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.935728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.935996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.936754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.938330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.948389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1184/2329] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:22:23.952511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.962504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aeb23c-63e9-4351-ad0e-d6f34292fe26 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.967665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:23.968326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:23.968638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:23.968915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:23.969672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:23.971274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b024e6f9-052a-4634-968b-a6aa97932005 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:23.981264 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1146/2330] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:22:23.985379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:23.995413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba25c2-d242-4f43-9058-d0eeb64c160c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:23.999934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:24.000834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:24.001115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:24.001388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] PATH_INFO: `/v3/projects/9b6cefef34ec4468b748ab41424762be/users/5cde7971e2474218b5acfa0318a8ad3f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:24.016407 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6cefef34ec4468b748ab41424762be, user_id=5cde7971e2474218b5acfa0318a8ad3f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:24.019096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:24.033201 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1185/2331] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:22:23 2022] PUT /identity/v3/projects/9b6cefef34ec4468b748ab41424762be/users/5cde7971e2474218b5acfa0318a8ad3f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:22:24.043163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f0ca-c657-4d9b-abba-298ec04d8d02 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:24.043453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f0ca-c657-4d9b-abba-298ec04d8d02 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:24.043728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992f0ca-c657-4d9b-abba-298ec04d8d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:24.072149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0992f0ca-c657-4d9b-abba-298ec04d8d02 None None] MFA Rules not processed for user `5cde7971e2474218b5acfa0318a8ad3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:24.072914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0992f0ca-c657-4d9b-abba-298ec04d8d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:24.148238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1147/2332] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:24.159238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:24.161367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:24.242998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7af1a-9bb2-40b9-9815-7a21ae8cdd93 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:24.247503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:24.248168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:24.248490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:24.248768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:24.258434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:24.259894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:24.270572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1186/2333] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:24.902333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96d98-ea31-420e-ab31-61a7dc9cfd67 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:24.903441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96d98-ea31-420e-ab31-61a7dc9cfd67 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:24.904036 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96d98-ea31-420e-ab31-61a7dc9cfd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:24.941733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c0a96d98-ea31-420e-ab31-61a7dc9cfd67 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:24.943517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c0a96d98-ea31-420e-ab31-61a7dc9cfd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:24.989349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:25.060509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1148/2334] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:22:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:25.081923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:25.083286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:25.146275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:25.173648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88596293-a844-4d50-86d4-271f83afe625 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:25.181037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:25.181964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:25.182428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:25.182966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:25.193949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:25.200749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:25.212664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1187/2335] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:22:26.844170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5d8d-a5c0-4fdf-9495-c997e7245b42 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:26.844689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5d8d-a5c0-4fdf-9495-c997e7245b42 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:26.845125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e5d8d-a5c0-4fdf-9495-c997e7245b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:26.882516 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5c5e5d8d-a5c0-4fdf-9495-c997e7245b42 None None] MFA Rules not processed for user `2b27ee7d15ca42bca6a09dd18751f01e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:26.884044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5c5e5d8d-a5c0-4fdf-9495-c997e7245b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:26.964336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1149/2336] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:35.208934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:36.226627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:36.362652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5a918-941f-49c6-a361-b65e73bce4f4 service swift] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:36.376224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:36.377691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:36.378699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:36.379196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] PATH_INFO: `/v3/users/5cde7971e2474218b5acfa0318a8ad3f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:36.380687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cde7971e2474218b5acfa0318a8ad3f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:36.382839 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:36.496221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cde7971e2474218b5acfa0318a8ad3f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:36.497038 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cde7971e2474218b5acfa0318a8ad3f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:36.497741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cde7971e2474218b5acfa0318a8ad3f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:36.512064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cde7971e2474218b5acfa0318a8ad3f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:36.537227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cde7971e2474218b5acfa0318a8ad3f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:36.583514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1188/2337] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:22:36 2022] DELETE /identity/v3/users/5cde7971e2474218b5acfa0318a8ad3f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:36.588092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:36.588642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:36.588870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:36.638065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:36.639420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:36.797262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1150/2338] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:22:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:36.841017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:36.843249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:36.971770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb8e0-1b9b-42b6-a97e-326f1bd18dce admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:36.977548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:36.978525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:36.979169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:36.979805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:36.998111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:37.000566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47190289-93af-42de-8cba-afcbc17fb1ce service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:37.023222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1189/2339] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:37.674339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:37.675933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:37.700149 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92811e-650a-4a70-b9ab-4928753c535a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:37.712179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:37.713242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:37.715298 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:37.715960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] PATH_INFO: `/v3/projects/9b6cefef34ec4468b748ab41424762be` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:37.717702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6cefef34ec4468b748ab41424762be)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:37.720017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:37.818850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6cefef34ec4468b748ab41424762be', 'request_id': 'req-e14207a4-7f32-44cf-90be-12966d5ca22d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:37.829280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6cefef34ec4468b748ab41424762be', 'request_id': 'req-e14207a4-7f32-44cf-90be-12966d5ca22d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:37.840556 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6cefef34ec4468b748ab41424762be', 'request_id': 'req-e14207a4-7f32-44cf-90be-12966d5ca22d'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:37.850336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:37.856545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] The token cache is being invalidate because project 9b6cefef34ec4468b748ab41424762be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:22:37.856545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:37.857668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e14207a4-7f32-44cf-90be-12966d5ca22d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:22:37.864365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1151/2340] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:22:37 2022] DELETE /identity/v3/projects/9b6cefef34ec4468b748ab41424762be => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:22:43.849680 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:43.850305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:43.850305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:43.851638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1190/2341] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:22:43.859115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabb9a-b951-4927-85a4-5952d4105c9b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:43.859615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabb9a-b951-4927-85a4-5952d4105c9b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:43.859722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eabb9a-b951-4927-85a4-5952d4105c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:43.910780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b0eabb9a-b951-4927-85a4-5952d4105c9b None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:43.911993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b0eabb9a-b951-4927-85a4-5952d4105c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.006778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1152/2342] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:22:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:44.018954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:44.020636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.047983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:44.127053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519b0e-9af8-4579-ab96-cc3e426abba1 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:44.133937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:44.135186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.135676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.136163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.150905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:44.153124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02335aaf-ccb5-45d6-8623-7bdf1dda2ed1 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:44.164011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1191/2343] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:44.228627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.229009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.229283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.230706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1153/2344] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:22:44.236022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e75361-2746-4d69-ac4e-d83021cc8555 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.236328 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e75361-2746-4d69-ac4e-d83021cc8555 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.236616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e75361-2746-4d69-ac4e-d83021cc8555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.254508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-46e75361-2746-4d69-ac4e-d83021cc8555 None None] MFA Rules not processed for user `6c28d40778004a2d8f2b8e2cb38a1f12`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:44.255700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-46e75361-2746-4d69-ac4e-d83021cc8555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.321254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1192/2345] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:22:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:44.333487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:44.335301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.351702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626c099-2a10-43db-a58b-ea42b4089ead None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:44.356855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:44.357631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.358180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.358456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.367422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:44.369096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c89c611-9351-431c-82c8-faeead206998 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:44.378843 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1154/2346] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:44.468983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.469168 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.469418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] PATH_INFO: `/` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.470921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1193/2347] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:22:44.476979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.477374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.477853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.498374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] MFA Rules not processed for user `aa1fb96699054e5c94fcc188fd9535be`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:44.499595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.508895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1155/2348] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:22:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:22:44.520762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:44.522479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:44.536480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ab729-85c3-4c76-aec3-41b52fa5e8cd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:44.540181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:44.540854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:44.541132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:44.541405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:44.552388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:44.553806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbfbd89-cd54-4178-a9e5-5048ccf017e4 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:44.564984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1194/2349] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:22:45.454103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:45.463394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ac497-dcfe-4562-b6c1-8c1700e18e46 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:45.467449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:45.468083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:45.468421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:45.468703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:45.470449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:45.534599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:45.536057 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:45.546532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1156/2350] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:22:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:22:49.520865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5e33-16d3-49e0-9396-32b2065c14f4 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:49.521344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5e33-16d3-49e0-9396-32b2065c14f4 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:49.521435 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5e33-16d3-49e0-9396-32b2065c14f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:49.570725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8b7f5e33-16d3-49e0-9396-32b2065c14f4 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:49.571991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8b7f5e33-16d3-49e0-9396-32b2065c14f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:49.639104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1195/2351] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:49.660794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:49.662358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:49.702183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:22:49.760268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f91bf-1215-4382-86ed-bb1aa4226e1b service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:49.766661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:49.767329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:49.767611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:49.767888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:49.781822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:49.783467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:49.793884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1157/2352] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:22:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:22:53.460026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:53.460467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:53.460685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:53.486615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:22:53.487955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:22:53.500604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1196/2353] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:22:53.514703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:22:53.525207 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21d4fe-c003-476c-b545-10e9a36b3fb8 service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:22:53.531141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:22:53.531807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:22:53.532110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:22:53.532444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:22:53.541519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:22:53.543685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e22c2-f861-4fa8-bb1f-6a6db8af65a1 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:22:53.552510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1158/2354] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:22:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:03.474754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:03.548747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=32 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:04.885436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:04.888059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:05.024068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd257994-8421-4a34-acb6-f8b4f04ebcd2 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:05.031302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:05.032167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:05.032550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:05.033060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:05.035176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:05.049891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:05.053068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:05.064410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1197/2355] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:09.272966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:09.273445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:09.273590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:09.291251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:09.292460 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:09.299421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1159/2356] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 15:23:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:09.312382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:09.324106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c32f92f-f548-4e49-852e-629a74fca045 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:09.329060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:09.329694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:09.329985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:09.330243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:09.338267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:09.339836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:09.347949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1198/2357] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:11.341411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:11.343049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:11.411796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd60f2-2b42-4ad7-8604-3585f7c29576 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:11.416222 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:11.417602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.417876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.418143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] PATH_INFO: `/v3/users/2b27ee7d15ca42bca6a09dd18751f01e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.419329 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b27ee7d15ca42bca6a09dd18751f01e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:11.421284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:11.465885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b27ee7d15ca42bca6a09dd18751f01e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.466545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b27ee7d15ca42bca6a09dd18751f01e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.467150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b27ee7d15ca42bca6a09dd18751f01e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.474141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b27ee7d15ca42bca6a09dd18751f01e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.486718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d7da3175-70f3-4286-9eef-befbdb86c34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b27ee7d15ca42bca6a09dd18751f01e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.500765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1160/2358] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:23:11 2022] DELETE /identity/v3/users/2b27ee7d15ca42bca6a09dd18751f01e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:11.672423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:11.683259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a8c2f-61b5-43c6-92e5-81d133910911 service swift] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:11.739477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:11.740256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.740567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.740841 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] PATH_INFO: `/v3/projects/f7c8c75b264644b7af9831e14cbf7841` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.741995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c8c75b264644b7af9831e14cbf7841)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:11.743886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:11.788353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c8c75b264644b7af9831e14cbf7841', 'request_id': 'req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.794400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c8c75b264644b7af9831e14cbf7841', 'request_id': 'req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.795994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:11.802312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c8c75b264644b7af9831e14cbf7841', 'request_id': 'req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.813959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] The token cache is being invalidate because project f7c8c75b264644b7af9831e14cbf7841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:23:11.814275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.814901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5af5a4ac-2e6c-45fe-8e65-c6899cfb9a11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:11.816482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1199/2359] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:23:11 2022] DELETE /identity/v3/projects/f7c8c75b264644b7af9831e14cbf7841 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:11.821279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.821569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.821848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.839417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:11.840648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:11.900204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1161/2360] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:11.908534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.908820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.909095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.929046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:11.930229 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:11.938931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1200/2361] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:23:11.943012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:11.952801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403197c-c688-4697-ace7-d11f378ea6df None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:11.957464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:11.958115 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.958399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.958677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.959398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:11.959705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:11.961388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:11.969817 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:11.975068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1162/2362] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:23:11.977981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:11.988054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2be48e2-9dc9-46ac-a1a3-96c2d4fc5067 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:11.992848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:11.993443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:11.993726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:11.994008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:11.994701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:11.997231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.022331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1201/2363] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:23:11 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:12.026546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.037131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188c144-97d0-4772-8db4-12bec5f27754 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.094310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.095017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.095304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.095584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.096285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.098315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.118316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1163/2364] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:23:12 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:12.123114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.133254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba739d-365e-46f8-8195-bded8b250c34 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.138401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.139054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.139335 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.139639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.140404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.142003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.152391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1202/2365] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:23:12.156072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.165977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caad330-b310-4ad8-86c2-2d0b42319aaa admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.170621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.171271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.171555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.171833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.172584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.174194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905454d1-48b2-4cbd-b617-4dafa8211800 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.184787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1164/2366] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:23:12.189093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.199726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb89969-908a-4a89-946f-c1a562db7598 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.204259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.205156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.205434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.205704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] PATH_INFO: `/v3/projects/81b988dbfce74357ae2383a2890d82b1/users/b6423cda89fb4897ae5d8bcba396e4d2/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.220825 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b988dbfce74357ae2383a2890d82b1, user_id=b6423cda89fb4897ae5d8bcba396e4d2, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.223508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.237465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1203/2367] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:23:12 2022] PUT /identity/v3/projects/81b988dbfce74357ae2383a2890d82b1/users/b6423cda89fb4897ae5d8bcba396e4d2/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:12.247958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.248197 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.248508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.276694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] MFA Rules not processed for user `b6423cda89fb4897ae5d8bcba396e4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:12.278012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:12.350185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1165/2368] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:12.357526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.359555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:12.426276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f851d17-9f3f-403a-bc0f-decbd7c5fba4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.430096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.430753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.431037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.431312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.441035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.442507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dfa78-5f7e-491c-805a-e7d3e8b48e9c service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.451953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1204/2369] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:12.478034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.550000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff67a1d-8c0a-4b26-a715-e22e39cbe09f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.554745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.555688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.556042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.556450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] PATH_INFO: `/v3/users/b6423cda89fb4897ae5d8bcba396e4d2` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.558251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6423cda89fb4897ae5d8bcba396e4d2)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.560273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.571808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:12.610337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6423cda89fb4897ae5d8bcba396e4d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:12.610992 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6423cda89fb4897ae5d8bcba396e4d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:12.611606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6423cda89fb4897ae5d8bcba396e4d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:12.618522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6423cda89fb4897ae5d8bcba396e4d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:12.630837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6423cda89fb4897ae5d8bcba396e4d2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:12.643557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1166/2370] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:23:12 2022] DELETE /identity/v3/users/b6423cda89fb4897ae5d8bcba396e4d2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:12.648138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.648464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.648759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.665467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:12.667175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:12.724665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1205/2371] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:12.735067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:12.736522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:12.792832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9950117-a771-456f-ae0a-b787b14a9879 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:12.796416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:12.797053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:12.797338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:12.797615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:12.806533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:12.808603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb5663-d2be-4028-a67d-198bc23a187f service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:12.817891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1167/2372] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:13.119670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.121589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:13.133508 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350bbd2-c344-4f13-82ca-572c85ca4b74 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.139001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.139873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.140159 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.140473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] PATH_INFO: `/v3/projects/81b988dbfce74357ae2383a2890d82b1` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.141677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b988dbfce74357ae2383a2890d82b1)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.143605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.167765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:13.192437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b988dbfce74357ae2383a2890d82b1', 'request_id': 'req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:13.198564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b988dbfce74357ae2383a2890d82b1', 'request_id': 'req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:13.205847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b988dbfce74357ae2383a2890d82b1', 'request_id': 'req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:13.223663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] The token cache is being invalidate because project 81b988dbfce74357ae2383a2890d82b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:23:13.223950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:13.224604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-dc235215-c26b-4bf6-89c6-bcc9d8d0b835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:13.226149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1206/2373] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:23:13 2022] DELETE /identity/v3/projects/81b988dbfce74357ae2383a2890d82b1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:13.232094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.232418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.232963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.250623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:13.251802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:13.310603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1168/2374] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:13.318010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.318301 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.318581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.338029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:13.339180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:13.347029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1207/2375] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:13.350988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.361582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae74667-f3ba-45f9-bd37-046f4748d940 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.366223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.366877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.367158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.367434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.368157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:13.368506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.370702 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.382049 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1169/2376] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:13.386155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.395138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1fc82-0156-42f7-b83c-52d4b79a324e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.399822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.400495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.400779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.401099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.401796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.403672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.433311 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1208/2377] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:23:13 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:23:13.434725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.445127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877eaf-6fa3-43f9-82f9-1a2b3499de31 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.457671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:13.507307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.507987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.508273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.508602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.509306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.511246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.531902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1170/2378] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:23:13 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:23:13.535988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.546975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2fe079-a110-4586-a758-a494a69ca1fd admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.551613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.552279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.552591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.552865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.553608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.555203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.566409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1209/2379] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:23:13.570428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.580750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af83c29-25fa-4dfe-8204-2f960978bfe5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.585694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.586751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.587086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.587459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.588267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.589899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccd974-e83a-42d3-b94b-cdf4815187d1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.600168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1171/2380] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:23:13.604722 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.615383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c9830-8f6e-4d5d-b06c-7cc2d8a5a4b5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.620583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.621472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.621775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.622044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] PATH_INFO: `/v3/projects/c71a42e029754231b2e00938acc6b7f3/users/e62db1d026f949eb82a425bc8dcd6377/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.636412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71a42e029754231b2e00938acc6b7f3, user_id=e62db1d026f949eb82a425bc8dcd6377, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.639037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.652107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1210/2381] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:23:13 2022] PUT /identity/v3/projects/c71a42e029754231b2e00938acc6b7f3/users/e62db1d026f949eb82a425bc8dcd6377/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:13.662250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08839d9-8759-41fa-ab02-47b85a616577 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.662640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08839d9-8759-41fa-ab02-47b85a616577 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.662929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08839d9-8759-41fa-ab02-47b85a616577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.690223 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c08839d9-8759-41fa-ab02-47b85a616577 None None] MFA Rules not processed for user `e62db1d026f949eb82a425bc8dcd6377`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:13.691416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c08839d9-8759-41fa-ab02-47b85a616577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:13.755952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1172/2382] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:13.767353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:13.769172 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:13.824582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496801-227f-46c7-b8a7-c8383dfebcc8 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:13.828211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:13.828874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:13.829164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:13.829439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:13.837605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:13.839583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:13.848217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1211/2383] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:15.026800 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33af8-c99f-4608-a9c6-5417fc86e2bb None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:15.027233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33af8-c99f-4608-a9c6-5417fc86e2bb None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:15.027353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f33af8-c99f-4608-a9c6-5417fc86e2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:15.044612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-41f33af8-c99f-4608-a9c6-5417fc86e2bb None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:15.045774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-41f33af8-c99f-4608-a9c6-5417fc86e2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:15.100775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1173/2384] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:15.113366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:15.114752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:15.165627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:15.187651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02546f9-d26b-4409-af9a-bc4e489484fd service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:15.190156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:15.190444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:15.190719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:15.192773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:15.193422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:15.193705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:15.193981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:15.205098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:15.206689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7447a6-be86-4c73-9f1c-e9ea9535fdcc service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:15.208578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] MFA Rules not processed for user `e62db1d026f949eb82a425bc8dcd6377`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:15.209767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:15.215634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1212/2385] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 15:23:15.217419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1174/2386] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:25.203252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:25.211686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:29.071904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6174b-40ee-43c7-b546-ffe4c31eba48 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.072409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6174b-40ee-43c7-b546-ffe4c31eba48 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.072514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d6174b-40ee-43c7-b546-ffe4c31eba48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.090689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d1d6174b-40ee-43c7-b546-ffe4c31eba48 None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:29.092091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d1d6174b-40ee-43c7-b546-ffe4c31eba48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:29.099431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1213/2387] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:29.111958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:29.123717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b22f782-965d-43ba-a7cc-fd5e35083d58 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:29.128567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:29.129203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.129485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.129761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.138507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:29.140175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:29.148165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1175/2388] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:29.652155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2114-b39a-4e82-ac74-20906bb4fb5d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.652481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2114-b39a-4e82-ac74-20906bb4fb5d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.652790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2114-b39a-4e82-ac74-20906bb4fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.671383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1e3d2114-b39a-4e82-ac74-20906bb4fb5d None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:29.672609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1e3d2114-b39a-4e82-ac74-20906bb4fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:29.679661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1214/2389] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:23:29.691332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:29.701954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76d44c-7564-4ee6-a531-1395515045e6 service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:29.707200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:29.707842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.708157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.708466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.717082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:29.718682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:29.726842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1176/2390] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:29.916103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1969be-2aee-4fe8-83c6-2ca4f468347f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.916430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1969be-2aee-4fe8-83c6-2ca4f468347f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.916709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1969be-2aee-4fe8-83c6-2ca4f468347f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.933931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2a1969be-2aee-4fe8-83c6-2ca4f468347f None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:29.935109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2a1969be-2aee-4fe8-83c6-2ca4f468347f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:29.942289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1215/2391] 10.222.0.45 () {58 vars in 1009 bytes} [Tue May 24 15:23:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:29.954267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:29.965451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd217e8-9aa3-4f10-87f2-cf9c9512c195 service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:29.970511 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:29.971175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:29.971459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:29.971737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:29.979768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:29.981426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:29.989801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1177/2392] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:30.300849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.301310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.301417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.319012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:30.320231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:30.327917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1216/2393] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 15:23:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:30.339579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:30.350008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ca739-795e-4889-8c52-ec6eb4ab4129 service swift] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:30.354965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:30.355595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:30.355879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:30.356155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:30.364426 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:30.366017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504a529-0c89-47eb-85f9-b739f95428cd service swift] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:30.374980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1178/2394] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:23:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:23:32.396574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.457998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ff2c-157f-4713-876e-a7134ad6dd3c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:32.462518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:32.463376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.463658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.463931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] PATH_INFO: `/v3/users/e62db1d026f949eb82a425bc8dcd6377` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.465659 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e62db1d026f949eb82a425bc8dcd6377)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:32.467582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:32.514410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e62db1d026f949eb82a425bc8dcd6377'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.515096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e62db1d026f949eb82a425bc8dcd6377'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.515705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e62db1d026f949eb82a425bc8dcd6377'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.522859 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e62db1d026f949eb82a425bc8dcd6377'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.530439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e62db1d026f949eb82a425bc8dcd6377'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.548629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1217/2395] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:23:32 2022] DELETE /identity/v3/users/e62db1d026f949eb82a425bc8dcd6377 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:32.553054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.553522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.553624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.571826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:32.573038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:32.634424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1179/2396] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:32.644505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.646123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:32.703511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb9874-81ab-4fd0-84f9-40205b382754 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:32.707151 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:32.707815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.708095 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.708397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.715969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:32.718611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:32.721041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2618bcd-5565-492e-83cb-83a0eaa500e7 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:32.732002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1218/2397] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:32.899224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:32.900706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:32.913385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e3449-12db-4145-8b95-a75bfa0a8064 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:32.918030 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:32.918887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:32.919169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:32.919444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] PATH_INFO: `/v3/projects/c71a42e029754231b2e00938acc6b7f3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:32.920745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71a42e029754231b2e00938acc6b7f3)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:32.922837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:32.971624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71a42e029754231b2e00938acc6b7f3', 'request_id': 'req-1bc300af-3359-44db-b5dd-11fc3eff4729'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.978258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71a42e029754231b2e00938acc6b7f3', 'request_id': 'req-1bc300af-3359-44db-b5dd-11fc3eff4729'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.986291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71a42e029754231b2e00938acc6b7f3', 'request_id': 'req-1bc300af-3359-44db-b5dd-11fc3eff4729'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.998416 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] The token cache is being invalidate because project c71a42e029754231b2e00938acc6b7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:23:32.998701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:32.999333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1bc300af-3359-44db-b5dd-11fc3eff4729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:23:33.001122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1180/2398] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:23:32 2022] DELETE /identity/v3/projects/c71a42e029754231b2e00938acc6b7f3 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:33.005838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.006231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.006630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.024529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:33.025796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:33.096949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1219/2399] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:33.105102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.105453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.105735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.126676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:33.127939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:33.135997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1181/2400] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:33.141217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.152904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34688e8-643c-4e62-bcf5-314e864d7087 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.157609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.158269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.158548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.158820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.159544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:23:33.159877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.161523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.173626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1220/2401] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:23:33.177897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.186951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de790c6-d9f8-4fcb-acff-2aa134d281a9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.191498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.192136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.192520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.192792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.193469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.195471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.221180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1182/2402] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:23:33 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:33.225169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.235179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e579db4-dccd-48f5-bdf1-6b9645105059 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.290975 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.291637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.291919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.292190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.292934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.295021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.315947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1221/2403] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:23:33 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:23:33.319948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.325981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:33.332358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ede61e-2f9b-4e71-a153-eb0f43553bc6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.338164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.338989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.339369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.339720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.340570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.342278 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.352888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1183/2404] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:23:33.356746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.366737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7941b-5b55-4e88-b078-e42d2540877b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.371795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.372478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.372759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.373030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.373747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.375333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2f85f-4ba6-44cf-9d04-d8ad6ee3d2e0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.389005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1222/2405] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:23:33.390302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.400320 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbc43a-439a-4aed-94ac-3d39bb9c84e4 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.405821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.406641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.406914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.407195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] PATH_INFO: `/v3/projects/44674730e48240da874cc5fb9662455d/users/e754dd9099b54fa4ac8debfbc4a645b9/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.422180 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44674730e48240da874cc5fb9662455d, user_id=e754dd9099b54fa4ac8debfbc4a645b9, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.424873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.437140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1184/2406] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:23:33 2022] PUT /identity/v3/projects/44674730e48240da874cc5fb9662455d/users/e754dd9099b54fa4ac8debfbc4a645b9/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:23:33.442251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151cd1-6002-4fee-bf16-44b737c04cef None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.442725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151cd1-6002-4fee-bf16-44b737c04cef None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.442950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9151cd1-6002-4fee-bf16-44b737c04cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.460652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e9151cd1-6002-4fee-bf16-44b737c04cef None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:33.461837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e9151cd1-6002-4fee-bf16-44b737c04cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:33.519077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1223/2407] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:23:33.528942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:33.530638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:33.588041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf0719-1e68-40af-85ed-fed0bcb25f75 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:33.592428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:33.593065 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:33.593346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:33.593619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:33.607458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:33.609209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:33.618652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1185/2408] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:34.421706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30251e-eab4-4d2c-8b1d-0b0f594bc536 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:34.421916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30251e-eab4-4d2c-8b1d-0b0f594bc536 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:34.422166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30251e-eab4-4d2c-8b1d-0b0f594bc536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:34.428001 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:34.453135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5c30251e-eab4-4d2c-8b1d-0b0f594bc536 None None] MFA Rules not processed for user `e754dd9099b54fa4ac8debfbc4a645b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:34.454358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5c30251e-eab4-4d2c-8b1d-0b0f594bc536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:34.530892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1224/2409] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:34.538998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:23:34.540427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:34.603978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8082e1-763e-43d9-9b5c-54c9c50b5e19 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:23:34.608985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:23:34.609643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:34.609926 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:34.610202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:34.619720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:23:34.621289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5a23f-5133-48d0-9bd9-2302f077f167 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:23:34.631824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1186/2410] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:23:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:23:35.998642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152661-c623-4710-a735-2f34e18baf18 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:35.999123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152661-c623-4710-a735-2f34e18baf18 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:35.999233 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc152661-c623-4710-a735-2f34e18baf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:36.016474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-fc152661-c623-4710-a735-2f34e18baf18 None None] MFA Rules not processed for user `e754dd9099b54fa4ac8debfbc4a645b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:36.017669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-fc152661-c623-4710-a735-2f34e18baf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:36.024633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1225/2411] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:23:44.626970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:46.011087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=34 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:23:49.962239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:23:49.962758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:23:49.962874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:23:49.980087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] MFA Rules not processed for user `2d8a7414fd254ec981d35b417dc373cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:23:49.981977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:23:50.037927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1187/2412] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:23:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:24:00.031411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=25 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:24:55.751600 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.752138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.752138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.769566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] MFA Rules not processed for user `2befe74ad06542b785d326101fc06d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:24:55.770768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:55.826016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1226/2413] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:24:55.861256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:55.862630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:55.919423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada9d31-0466-42e4-97b0-629cdcc2e98e None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:55.923002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:55.923646 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:55.923933 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:55.924212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:55.926043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:56.000046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.001534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.013506 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1188/2414] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:24:56.614117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:24:56.624871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91bd7c-cd1d-453d-a7f3-856a0f05a4fe None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:24:56.629777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:24:56.630484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:24:56.630827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:24:56.631187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:24:56.633029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:24:56.646992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:24:56.648459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:24:56.660103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1227/2415] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:25:04.479280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:04.480831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:04.494617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac444b5c-e0df-41da-901a-db378cbf215e service glance] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:04.500013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:04.500670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:04.500949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:04.501221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:04.502879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:04.513289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:04.514895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:04.524045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1189/2416] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:25:06.654772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=31 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:14.519723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=60 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:17.033490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:17.034909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:17.049193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025673f8-4085-4d3d-9fd2-da259bd0d414 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:17.054693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:17.055324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:17.055608 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:17.055887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:17.057612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:17.116149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:17.117784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:17.127526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1228/2417] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:25:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:25:27.123316 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=28 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:25:43.609408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:25:43.621421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f669-e27b-4b60-accf-a64d19e7623c service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:25:43.626905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:25:43.627538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:25:43.627820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:25:43.628095 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:25:43.629898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:25:43.640139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:25:43.641781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:25:43.651161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1190/2418] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:25:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:25:53.646611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:06.691931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:06.701375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6eebe-8d5f-4179-95a3-ad5f078aea3e service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:06.704979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:06.705617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:06.705898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:06.706174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:06.707943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:06.717324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:06.718737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:06.726531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1229/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:26:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:26:16.723018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:26:41.301452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:26:41.312568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759de36a-ae71-4755-befa-8becbce7672b service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:26:41.318123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:26:41.318674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:26:41.318951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:26:41.319221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:26:41.321099 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:26:41.380138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:26:41.381791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:26:41.390788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1191/2420] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:26:51.386562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=28 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:06.054634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:06.056012 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:06.127050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad26a8-0ba0-44d9-b5c9-3930b201e659 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:06.132010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:06.132690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:06.132972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:06.133245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:06.135003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:06.146876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:06.148487 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:06.158545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1230/2421] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:28:16.154205 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=32 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:28:34.771148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:28:34.773393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:28:34.785289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c79de-3bf2-4bf6-9c41-a98d2414a8f6 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:28:34.789231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:28:34.789863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:28:34.790145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:28:34.790421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:28:34.798447 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:28:34.799889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:28:34.809178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1192/2422] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:28:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:28:44.804588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:04.795758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:04.807895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba72a79-2f63-494f-8bdd-87e831b44146 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:04.813389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:04.814028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:04.814313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:04.814590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:04.822912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:04.824554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:04.851744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1231/2423] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:30:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:13.395716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:13.404958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26d7c1-bb8f-4aab-8aba-fb96c1d1e578 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:13.408532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:13.409166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:13.409446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:13.409720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:13.420654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:13.422108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:13.433216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1193/2424] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:30:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:14.849130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=13 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:16.768714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:16.780577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c1443-fcd1-457e-9725-1572662e56bb service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:16.784118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:16.784790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:16.785073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:16.785348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:16.793602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:16.795017 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:16.821772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1232/2425] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:30:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:23.428375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:26.818706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=13 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:29.316751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:29.325741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d246757-77e3-4fd4-9e56-5b0b0c5b7c0d service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:29.329418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:29.330047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:29.330324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:29.330591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:29.340784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:29.342213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:29.353458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1194/2426] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:30:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:39.348071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:30:42.639337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:30:42.647840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d4cc1-0e1a-46e8-9037-0cfb28e78c69 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:30:42.651366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:30:42.651990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:30:42.652267 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:30:42.652569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:30:42.660961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:30:42.662445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:30:42.690570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1233/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:30:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:30:52.687872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=13 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:13.652942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:13.662257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4136a-4fb4-4c56-b9c0-716ae7b69f9e service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:13.665809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:13.666442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:13.666732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:13.667001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:13.674275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:13.675723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:13.683619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1195/2428] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:31:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:31:23.680048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:41.221536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:41.229933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031965e-641f-480f-94bb-98fa0fda79a7 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:41.233486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:41.234120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:41.234395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:41.234664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:41.243502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:41.244951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:41.271591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1234/2429] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:31:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:45.328911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:31:45.342920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d35739-31d0-45ce-9e0a-14d780557c9a service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:31:45.348936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:31:45.349975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:31:45.351186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:31:45.351923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:31:45.355555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:31:45.376314 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:31:45.379137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:31:45.397482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1196/2430] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:31:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:31:51.269109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=13 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:31:55.389891 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:33:10.124410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:33:10.183833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f3a2c-0d65-4c80-aedb-45e6761465fd service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:33:10.189182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:33:10.189943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:33:10.190098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:33:10.190378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:33:10.199251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:33:10.200865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:33:10.213118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1235/2431] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:33:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:33:20.206570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=27 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:33:34.963710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:33:35.045158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb2dd74-8906-411e-b209-969b0d2d4492 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:33:35.048682 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:33:35.049316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:33:35.049592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:33:35.049887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:33:35.051733 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:33:35.140074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:33:35.141561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:33:35.158957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1197/2432] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:33:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:33:45.156524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=78 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:35:03.914408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:35:03.923594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268867c4-3203-4591-ac15-54738fb62dd8 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:35:03.927808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:35:03.928473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:35:03.928751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:35:03.929023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:35:03.984004 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:35:03.985460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:35:03.995052 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1236/2433] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:35:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:35:13.990987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=28 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:35:16.358832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:35:16.368806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e1475e-481d-4303-8b1b-f9283c9f2d00 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:35:16.372411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:35:16.373079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:35:16.373355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:35:16.373626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:35:16.383840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:35:16.385475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:35:16.394725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1198/2434] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:35:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:35:26.390513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:35:42.785797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:35:42.794592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c72541-92cd-417e-9a7e-a49c4b8548ac service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:35:42.798236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:35:42.798880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:35:42.799164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:35:42.799444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:35:42.855239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:35:42.856746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:35:42.865977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1237/2435] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:35:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:35:52.861528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=28 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:08.209961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:08.211527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:08.287957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a304ea75-d340-45c0-a1e9-af79fc971987 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:08.291633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:08.292272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:08.292583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:08.292858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:08.294741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:08.381257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:08.382713 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:08.403035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1199/2436] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:36:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:36:09.587320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:09.588775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:09.601556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe77cb1-856d-4b4b-8a67-738289e84167 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:09.605149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:09.605798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:09.606082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:09.606360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:09.608220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:09.624474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:09.626252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:09.638479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1238/2437] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:36:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:18.400452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=75 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:19.632584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:36:20.642745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:36:20.651686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78c0e-890a-4b06-95f2-441e30c476e2 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:36:20.655305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:36:20.655939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:36:20.656221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:36:20.656528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:36:20.658381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:36:20.667711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:36:20.669170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:36:20.677151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1200/2438] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:36:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:36:30.673538 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:10.136101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:10.145396 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39978dc5-a3ca-4087-b5d8-c25f07694181 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:10.149227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:10.149854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:10.150131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:10.150401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:10.152226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:10.222450 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:10.223884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:10.233214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1239/2439] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:38:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:14.058944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:14.060455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.146363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9720c3-8840-4983-a03d-439f2002e5c8 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:14.151491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:14.152146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:14.152459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:14.152740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:14.154631 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:14.167092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:14.168753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:14.179161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1201/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:38:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:20.228698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=33 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:24.174654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=37 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:38:35.958156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:38:35.959781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:38:35.975674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686f873b-8726-4aff-95ce-76f0002f73a4 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:38:35.981145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:38:35.981785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:38:35.982069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:38:35.982348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:38:35.991028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:38:35.992639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:38:36.001401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1240/2441] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:38:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:38:45.997360 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:04.060945 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:04.073490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5391f33-43c9-462c-a9af-e91caa95cdc0 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:04.079014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:04.079658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:04.079934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:04.080205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:04.082607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:04.105948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:04.107529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:04.116701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1202/2442] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:40:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:14.112260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=11 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:40:16.387570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:40:16.389185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:16.403573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d317e6-7005-4279-8d4f-79d740926da6 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:40:16.409058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:40:16.409696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:40:16.409979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:40:16.410254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:40:16.412585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:40:16.483812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:40:16.486178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:40:16.496858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1241/2443] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:40:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:40:26.492313 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=33 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:08.098549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:08.107785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f108c7-a40c-4791-a0ac-18f50d7c9104 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:08.111356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:08.111987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:08.112265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:08.112596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:08.123485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:08.125196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:08.136832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1203/2444] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:41:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:08.982383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:08.991618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14b7a5-d96d-445e-9b3a-ed67ad091d40 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:08.995668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:08.996391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:08.996606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:08.996879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:09.006958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:09.008741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:09.019949 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1242/2445] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:41:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:18.131181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:19.015044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:41:24.717629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:41:24.726465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56eea3c-910c-48ff-b7db-e92337b4624b service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:41:24.730685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:41:24.731317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:41:24.731598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:41:24.731874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:41:24.739400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:41:24.740883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55109298-d21e-4537-b7fe-229d6855f926 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:41:24.749124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1204/2446] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:41:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:41:34.745485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:10.628765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:10.629219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:10.654079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6ce6f-e124-47e2-abdf-0abf7621524a service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:10.657648 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:10.658317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:10.658603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:10.658877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:10.660893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:10.694686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:10.696133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:10.715402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1243/2447] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:42:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:10.967435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:10.967973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:10.967973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:11.004764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] MFA Rules not processed for user `40db5795ce7e4c5bbf06fb81ce404e78`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:42:11.005982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:11.024747 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1205/2448] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:42:11.357294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:11.366816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f609a-7767-4c88-bbad-51b4a3aa5c66 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:11.370442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:11.371106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:11.371387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:11.371663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:11.373461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:11.383568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:11.385019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:11.393500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1244/2449] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:42:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:11.715822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:11.728164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7dcb81-7305-49e0-8e4d-f2630d23c1e1 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:11.733211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:11.733893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:11.734174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:11.734453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:11.736217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:11.748844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:11.750448 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:11.764973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1206/2450] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:42:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:14.398523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:14.422726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a448a92-82a7-4096-afcc-dc1a66309c33 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:14.440014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:14.441138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:14.442196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:14.442515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:14.444704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:14.465790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:14.467802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:14.491861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1245/2451] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:42:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:19.547844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:42:19.560324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c0b95-ad19-47d1-b620-0c314cdc02de service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:42:19.565754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:42:19.566402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:42:19.566684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:42:19.566959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:42:19.568754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:42:19.581652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:42:19.583279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:42:19.596003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1207/2452] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:42:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:42:24.476337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=36 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:42:29.591380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=26 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:09.228823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:09.239995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da26b0-04fe-4f51-8d62-f07979970a20 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:09.245806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:09.246441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:09.246724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:09.247000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:09.254924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:09.256528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:09.284276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1246/2453] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:43:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:19.065216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:43:19.066775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:43:19.111788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda48382-e109-46dc-ac6f-f39439d36d26 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:43:19.117060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:43:19.117709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:43:19.117993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:43:19.118266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:43:19.127364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:43:19.128997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56c95eb-9c7c-4bcc-a868-e8c7f6f703c9 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:43:19.138670 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1208/2454] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:43:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:43:19.281657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=13 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:43:29.134530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=19 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:10.825457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:10.911787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afa596-7e46-498a-9f30-4a901100c8fe service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:10.915827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:10.916516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:10.916802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:10.917100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:10.919141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:11.018701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:11.020144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:11.039737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1247/2455] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:11.283141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:11.284166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:11.284672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:11.356887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] MFA Rules not processed for user `c1d39def9fcb44948c25c9139d2138b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:11.358221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:11.437703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1209/2456] 10.222.0.45 () {58 vars in 967 bytes} [Tue May 24 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:11.522572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:11.525288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:11.599624 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:11.657907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4549e2fa-aeb3-4b49-9cbe-f027d9138e5b service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:11.669943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec47b745ff5a4ced9b47ed5ecd2b28cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:11.671049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:11.671436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:11.671830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:11.682662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:11.684359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:11.696002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1248/2457] 10.222.0.45 () {60 vars in 1384 bytes} [Tue May 24 15:44:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 15:44:12.266384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:12.277434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbafef2-27a5-4e0b-8583-85669972b76a None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:12.281471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:12.282468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:12.282939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:12.283347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:12.287020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:12.379880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:12.381408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a614dfe-cf68-4d89-bf7d-d01737b760ed service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:12.403472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1210/2458] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:15.498885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:15.508166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c022289-de33-4718-aa64-4f43f0163e2d service swift] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:15.511711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:15.512403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:15.512685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:15.512961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:15.514705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:15.527025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:15.528478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb304c5-47ba-4f86-84fa-c01ec4030508 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:15.538351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1249/2459] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:18.789643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:18.790768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:18.790875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] PATH_INFO: `/` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:18.792342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1211/2460] 10.222.0.45 () {54 vars in 921 bytes} [Tue May 24 15:44:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 15:44:18.821363 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:18.822187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:18.822472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:18.864851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:18.867579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:18.879437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] MFA Rules not processed for user `1ac56ca4fe95468784f955de52375920`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:18.880669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:18.884719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1b80b-957b-42da-80c2-da0a6d694282 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:18.889352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:18.890065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1250/2461] 10.222.0.45 () {58 vars in 1018 bytes} [Tue May 24 15:44:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:18.890396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:18.890684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:18.890956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] PATH_INFO: `/v3/users/2befe74ad06542b785d326101fc06d0b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:18.892161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2befe74ad06542b785d326101fc06d0b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:18.894132 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:18.902208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:18.912982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74e7e3-21e6-4c77-a30e-d0a80d3c9d98 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:18.918179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:18.918418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:18.919383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:18.919898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:18.920033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:18.930376 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:18.932642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:18.942460 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1251/2462] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:44:18.945005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2befe74ad06542b785d326101fc06d0b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:18.945698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2befe74ad06542b785d326101fc06d0b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:18.946867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2befe74ad06542b785d326101fc06d0b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:18.954240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2befe74ad06542b785d326101fc06d0b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:18.967715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35e71e50-5ad9-434b-9af5-0778d4db4513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2befe74ad06542b785d326101fc06d0b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:18.983023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1212/2463] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:44:18 2022] DELETE /identity/v3/users/2befe74ad06542b785d326101fc06d0b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:44:19.161147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.172889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5749a-ab80-4527-ba4e-cd4d5bc8b5d9 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.233059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.233935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.234216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.234492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] PATH_INFO: `/v3/projects/0812b13a287c43259d7e09b0c658d4c8` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.235694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0812b13a287c43259d7e09b0c658d4c8)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.237709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.286440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0812b13a287c43259d7e09b0c658d4c8', 'request_id': 'req-2f0f44f6-6723-4e40-8710-0ee98d074ce0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:19.291863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0812b13a287c43259d7e09b0c658d4c8', 'request_id': 'req-2f0f44f6-6723-4e40-8710-0ee98d074ce0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:19.299310 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0812b13a287c43259d7e09b0c658d4c8', 'request_id': 'req-2f0f44f6-6723-4e40-8710-0ee98d074ce0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:19.300545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:19.313696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] The token cache is being invalidate because project 0812b13a287c43259d7e09b0c658d4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:44:19.313984 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:19.314611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2f0f44f6-6723-4e40-8710-0ee98d074ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:44:19.316142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1252/2464] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:44:19 2022] DELETE /identity/v3/projects/0812b13a287c43259d7e09b0c658d4c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:44:19.322784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.323161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.323440 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.367967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:19.369703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:19.429571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1213/2465] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:44:19.437955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.438303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.438519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.457481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:19.458675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:19.468917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1253/2466] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:19.473004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.482664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dfcd23-9096-4be5-91d4-c9c0e6a3daf1 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.487170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.487816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.488096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.488399 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.489133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:44:19.489439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.491048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.503089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1214/2467] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:44:19.507109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.517049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28185200-e62b-4421-a78b-ab269b08bafb None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.521688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.522332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.522619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.522899 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.523599 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.525535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.559853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1254/2468] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:44:19 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:44:19.563676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.574193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1397a4-fdc1-4c2a-9842-081236c2cc4d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.631302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.631976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.632258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.632562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.633253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.635265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.644778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:19.656003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1215/2469] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:44:19 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:44:19.660081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.670942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd306ae8-b48a-473f-9c3c-b48e608e77d5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.675529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.676184 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.676506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.676790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.677522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.679126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.689602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1255/2470] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:44:19.693123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.702904 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729f097c-7d9a-4a55-a1c0-31ecc8858381 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.707442 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.708084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.708409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.708688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.709405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.711002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a731a60-d6e7-47be-851b-8c18bc52ae51 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.720779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1216/2471] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:44:19.724568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.736982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf4ff6-8f65-4ae3-89ba-011b317ff0ad admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.744442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.745770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.746192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.746598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] PATH_INFO: `/v3/projects/3c965241379a45d8b9103af1913791a7/users/ca179fc319904fcb9b343c9e7de28e6f/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.767434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c965241379a45d8b9103af1913791a7, user_id=ca179fc319904fcb9b343c9e7de28e6f, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.770174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.783944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1256/2472] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:44:19 2022] PUT /identity/v3/projects/3c965241379a45d8b9103af1913791a7/users/ca179fc319904fcb9b343c9e7de28e6f/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:44:19.788963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd069-d113-4ef2-955a-1b81923af2c3 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.789255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd069-d113-4ef2-955a-1b81923af2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.789532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabd069-d113-4ef2-955a-1b81923af2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.807879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-bdabd069-d113-4ef2-955a-1b81923af2c3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:19.809094 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bdabd069-d113-4ef2-955a-1b81923af2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:19.866857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1217/2473] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:19.877141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:19.878858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:19.934945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a6291-c662-4041-9b6a-946adc3690e0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:19.938521 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:19.939150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:19.939427 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:19.939749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:19.948467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:19.949890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:19.960637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1257/2474] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:24.093128 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7177-eebe-4e86-b460-f495e187027c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:24.093674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7177-eebe-4e86-b460-f495e187027c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:24.093913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc7177-eebe-4e86-b460-f495e187027c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:24.122845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d5fc7177-eebe-4e86-b460-f495e187027c None None] MFA Rules not processed for user `ca179fc319904fcb9b343c9e7de28e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:24.124215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d5fc7177-eebe-4e86-b460-f495e187027c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:24.212867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1218/2475] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:24.227884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:24.230783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:24.313560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367400e0-a5cf-4716-8e06-c0e71b1eaca8 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:24.319594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:24.320483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:24.320773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:24.321156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:24.329761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:24.331382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:24.340964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1258/2476] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:26.647634 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:26.648070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:26.648165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:26.673582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] MFA Rules not processed for user `ca179fc319904fcb9b343c9e7de28e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:44:26.674804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:26.682566 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1219/2477] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:44:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:44:27.038353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:27.052604 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc633c-ca1e-40f2-8de4-45623bb5bf81 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:27.058666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:27.059526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:27.059869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:27.060163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:27.061983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:27.073553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:27.076580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:27.078302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:27.090489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1259/2478] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:36.666270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=69 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:37.091175 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:44:38.107299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:44:38.120729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc4b753-3b9e-42a7-b376-5cdb3620c6f1 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:44:38.126438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:44:38.127108 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:44:38.127392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:44:38.127666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:44:38.129557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:44:38.142633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:44:38.144408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:44:38.156644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1220/2479] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:44:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:44:48.152367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:46:11.745343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:11.746716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:11.803577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230ef99-3a47-44aa-b188-8bae77730b76 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:11.807156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:11.807811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:11.808092 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:11.808394 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:11.810257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:11.870816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:11.872262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:11.881865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1260/2480] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:46:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:11.930023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:11.939308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13fb75-8c53-466f-a231-ed5c15f36756 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:11.942912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:11.943543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:11.943821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:11.944145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:11.945922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:12.012745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:12.014129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:12.032504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1221/2481] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:46:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:14.572672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:14.591959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc5102-182a-4b48-8d22-cda2f6f43156 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:14.595549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:14.596192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:14.596568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:14.596846 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:14.598584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:14.630342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:14.631759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475ecb3-5d02-4c0c-8b84-f9e2f8f02244 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:14.652637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1261/2482] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:46:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:15.828861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:15.830292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:15.854567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1caedb4-d1a8-4fe9-82f1-2722da55edd4 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:15.858126 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:15.858760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:15.859043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:15.859318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:15.861101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:15.875148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:15.876568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:15.887392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1222/2483] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:46:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:20.252351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:20.252886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:20.252886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:20.270378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] MFA Rules not processed for user `ca179fc319904fcb9b343c9e7de28e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:46:20.271585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:20.278340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1262/2484] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:46:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:46:24.253303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:46:24.262098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d70b5-1a06-41e1-8455-abe440d36031 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:46:24.265857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:46:24.266505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:46:24.266786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:46:24.267070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:46:24.268861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:46:24.279290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:46:24.281173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:46:24.290450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1223/2485] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:46:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:46:30.265566 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=73 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:46:34.286735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=53 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:11.590480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:11.603471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123f8735-39ef-47f6-8c68-6f31d45d93bd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:11.608869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:11.609570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:11.609854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:11.610131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:11.611991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:11.625689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:11.627287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca15ce3-04e2-49b8-9ca8-9234c9eda85e service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:11.639784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1263/2486] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:12.604865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:12.614783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75450729-9700-47cb-82f8-ae2715590da9 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:12.619303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:12.620176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:12.620688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:12.620970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:12.622912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:12.707515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:12.709867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:12.721263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1224/2487] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:14.279002 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:14.279504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:14.279617 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:14.296520 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] MFA Rules not processed for user `ca179fc319904fcb9b343c9e7de28e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:48:14.297698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:14.304357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1264/2488] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:48:14.601407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:14.610585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8622-6150-4f45-a58a-d20e0fe14aaa service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:14.614244 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:14.614795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:14.615075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:14.615346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:14.617389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:14.627145 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:14.628651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:14.637159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1225/2489] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:14.936708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:14.938130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:15.021727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a8c69-89c2-46e1-84e5-94fc0bafdfd6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:15.027334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:15.027974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:15.028256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:15.028565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:15.030351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:15.041498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:15.043075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:15.053117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1265/2490] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:22.994499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:23.006862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e14771-70cc-4ddc-85b2-e8564264df90 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:23.011822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:23.012490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:23.012770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:23.013042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:23.014783 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:23.026309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:23.028397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:23.038290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1226/2491] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:23.459015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:48:23.473471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b677a32-ad09-4cff-b234-33cbd54b592c service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:48:23.478868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:48:23.479515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:48:23.479798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:48:23.480075 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:48:23.481861 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:48:23.494353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:48:23.495924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:48:23.506597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1266/2492] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:48:33.034188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=45 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:48:33.502101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=52 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:04.183790 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:04.185311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:04.200083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cfa40-d433-4160-98ee-d2d473adc211 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:04.205717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:04.206346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:04.206629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:04.206897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:04.214972 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:04.216577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:04.225121 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1227/2493] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:08.668254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:08.677332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5060abf-62fb-4bb2-81fc-248a7cbcceb8 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:08.680851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:08.681479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:08.681757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:08.682026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:08.690683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:08.692099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:08.701370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1267/2494] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:11.989488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:11.991118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.011203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdab3cc-0463-48cf-9a5b-18bee93fc4c6 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.017455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.018189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.018541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.018889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.032930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.054584 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.059219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.059857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.075654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1228/2495] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:12.084523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71062c0a-64fe-47b3-aac9-00680bbf2f55 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.090049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.090933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.091223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.091500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] PATH_INFO: `/v3/users/ca179fc319904fcb9b343c9e7de28e6f` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.092743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca179fc319904fcb9b343c9e7de28e6f)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.094674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.145550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca179fc319904fcb9b343c9e7de28e6f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.146334 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca179fc319904fcb9b343c9e7de28e6f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.146955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca179fc319904fcb9b343c9e7de28e6f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.154622 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca179fc319904fcb9b343c9e7de28e6f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.162342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-907d9116-0130-4dae-9357-6bce3ea26286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca179fc319904fcb9b343c9e7de28e6f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.179986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1268/2496] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:50:12 2022] DELETE /identity/v3/users/ca179fc319904fcb9b343c9e7de28e6f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:50:12.356136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.367510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03baea15-e9b6-4a20-9cae-6e146f6f31e6 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.428953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.429812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.430089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.430358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] PATH_INFO: `/v3/projects/3c965241379a45d8b9103af1913791a7` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.432005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c965241379a45d8b9103af1913791a7)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.433943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.479282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c965241379a45d8b9103af1913791a7', 'request_id': 'req-3e066b84-0eb4-4332-8bdf-5e4947e8752b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.484673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c965241379a45d8b9103af1913791a7', 'request_id': 'req-3e066b84-0eb4-4332-8bdf-5e4947e8752b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.491587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c965241379a45d8b9103af1913791a7', 'request_id': 'req-3e066b84-0eb4-4332-8bdf-5e4947e8752b'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.503234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] The token cache is being invalidate because project 3c965241379a45d8b9103af1913791a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:12.503517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.504136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3e066b84-0eb4-4332-8bdf-5e4947e8752b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:12.505667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1229/2497] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:50:12 2022] DELETE /identity/v3/projects/3c965241379a45d8b9103af1913791a7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:12.511096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.511391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.511698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.529154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:12.530340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.587729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1269/2498] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:12.595242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.595557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.595848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.613509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:12.614611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.623102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1230/2499] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:12.626983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.637330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95752233-21f7-4ec9-b57f-4a81c7b66654 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.641816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.642464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.642746 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.643020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.643745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:50:12.644053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.645688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.661999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1270/2500] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:50:12.663067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.672650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7e2d3-a2b8-475c-ac96-48de97e4d547 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.677401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.678056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.678335 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.678607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.679318 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.681917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.709451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1231/2501] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:50:12 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:12.710535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.720491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9722cc7-9dd2-432e-926c-e03eed7d726a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.765128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:12.780132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.780976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.781130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.781401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.782093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.783994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.804888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1271/2502] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:50:12 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:12.809583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.819722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9aa21b-a264-4b96-a250-51f68cc0c4c2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.824644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.825304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.825582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.825853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.826572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.828167 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.838502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1232/2503] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:12.842691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.848190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.849883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.854003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575abff3-ac7d-44e7-94d8-4686e68e91e5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.858773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.859540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.859831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.860117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.860888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.862547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.874153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1272/2504] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:12.877985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:12.879125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:12.887854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a4edb-e766-45a5-9084-c7843b93eebb admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.892503 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.893646 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.893929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.894197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] PATH_INFO: `/v3/projects/74188d6a458443efa85527510fdc1ac1/users/ef3c7fa879774e8d89717ac9ce478ef2/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.908810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=74188d6a458443efa85527510fdc1ac1, user_id=ef3c7fa879774e8d89717ac9ce478ef2, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.911015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609a246-e5fd-4d85-8484-3dfbd331f7fe admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:12.911478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e021a45-c284-4ee9-bce9-753060d36cdc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.914930 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:12.916279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.916592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.916868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.918749 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.924594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1273/2505] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:12 2022] PUT /identity/v3/projects/74188d6a458443efa85527510fdc1ac1/users/ef3c7fa879774e8d89717ac9ce478ef2/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:12.935243 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:12.935526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:12.935851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:12.961524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] MFA Rules not processed for user `ef3c7fa879774e8d89717ac9ce478ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:12.962749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:12.982419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:12.983875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:12.993184 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1233/2506] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:50:13.029777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1274/2507] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:50:13.363542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:13.365010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:13.425175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3400ef-841d-4e16-b262-a1af13371e0f service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:13.471864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:13.472529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:13.472809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:13.473080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:13.474907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:13.522153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:13.537669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:13.539101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:13.549259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1234/2508] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:16.391000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:16.392688 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:16.406901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f36f1c-fa13-4464-bc95-59b84de04df1 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:16.411303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:16.412169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.412486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.412749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] PATH_INFO: `/v3/users/40db5795ce7e4c5bbf06fb81ce404e78` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.413917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40db5795ce7e4c5bbf06fb81ce404e78)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:16.415819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:16.464332 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40db5795ce7e4c5bbf06fb81ce404e78'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.465011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40db5795ce7e4c5bbf06fb81ce404e78'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.465630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40db5795ce7e4c5bbf06fb81ce404e78'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.472371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40db5795ce7e4c5bbf06fb81ce404e78'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.479209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-916a9358-5943-4998-97a8-bb863ac13cc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40db5795ce7e4c5bbf06fb81ce404e78'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.496128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1275/2509] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:50:16 2022] DELETE /identity/v3/users/40db5795ce7e4c5bbf06fb81ce404e78 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:16.666489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:16.677705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4607f4-6718-44f0-8777-ec953ad44220 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:16.735337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:16.736195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.736505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.736776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] PATH_INFO: `/v3/projects/0fe1d758c40542e9b237f9d7bcfa5755` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.737948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe1d758c40542e9b237f9d7bcfa5755)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:16.739855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:16.785446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe1d758c40542e9b237f9d7bcfa5755', 'request_id': 'req-8576bba7-811c-48d1-a42f-bf335894daab'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.790875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe1d758c40542e9b237f9d7bcfa5755', 'request_id': 'req-8576bba7-811c-48d1-a42f-bf335894daab'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.797978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe1d758c40542e9b237f9d7bcfa5755', 'request_id': 'req-8576bba7-811c-48d1-a42f-bf335894daab'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.809004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] The token cache is being invalidate because project 0fe1d758c40542e9b237f9d7bcfa5755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:16.809289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.809898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:16.811391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1235/2510] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:50:16 2022] DELETE /identity/v3/projects/0fe1d758c40542e9b237f9d7bcfa5755 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:16.817100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64960-0198-444b-9e54-c3123c2c2080 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.817384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64960-0198-444b-9e54-c3123c2c2080 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.817656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64960-0198-444b-9e54-c3123c2c2080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.824457 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:16.838135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b5b64960-0198-444b-9e54-c3123c2c2080 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:16.839319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b5b64960-0198-444b-9e54-c3123c2c2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:16.867869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:16.869637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:16.906935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1276/2511] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:16.915438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.915652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.915929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.934279 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:16.934871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8576bba7-811c-48d1-a42f-bf335894daab admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:16.935539 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:16.938680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:16.939431 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.939710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.939979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.941754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:16.943767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1277/2512] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:50:16.948430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:16.957836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2473bd-a210-4a1e-85fb-ed06d8aae4e3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:16.962633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:16.963395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:16.963686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:16.964077 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:16.964836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:50:16.965143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:16.966804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:16.973169 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:16.978877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1278/2513] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:50:16.985221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:16.995590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397a179-50c4-43f2-bf1d-8c0f67537ec1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.000362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.000999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.001280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.001601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.002276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.004160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.005900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.007357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.017939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1236/2514] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:50:17.035963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1279/2515] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:50:16 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:17.037000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:17.047452 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be7b8-5e35-4922-9b36-55a771e5ea4f service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.111885 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.112587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.112874 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.113148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.113983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.115921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.138976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1237/2516] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:50:17 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:17.144069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:17.156524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5203d937-0c15-40f2-84e0-f861c12ef4a5 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.162161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.162640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.162923 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.163199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.163936 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.166320 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.177904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1280/2517] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:17.182240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:17.192577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29672c49-eee1-46ac-90eb-fe725dc08af9 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.197313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.197977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.198258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.198534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.199257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.201155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2250701-21e4-47e3-8ebe-2a02ceeb6739 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.211712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1238/2518] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:17.216114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:17.226779 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882f242-0411-429b-995f-98b811c57328 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.232479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.233495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.233869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.234210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] PATH_INFO: `/v3/projects/7f6a5b6aab4d44fd8c06968e7eb2117d/users/0c88621a24264fbf9efbdc9db1abdb5b/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.253480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f6a5b6aab4d44fd8c06968e7eb2117d, user_id=0c88621a24264fbf9efbdc9db1abdb5b, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.265582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.282348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1281/2519] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:17 2022] PUT /identity/v3/projects/7f6a5b6aab4d44fd8c06968e7eb2117d/users/0c88621a24264fbf9efbdc9db1abdb5b/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:50:17.284273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07a77-e4b0-4023-9566-4d1d1b5f51e6 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.284654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07a77-e4b0-4023-9566-4d1d1b5f51e6 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.285001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb07a77-e4b0-4023-9566-4d1d1b5f51e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.304471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2bb07a77-e4b0-4023-9566-4d1d1b5f51e6 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:17.306569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2bb07a77-e4b0-4023-9566-4d1d1b5f51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:17.392232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1239/2520] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:17.402388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:17.404417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:17.465330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10437a8f-8beb-450b-9905-a3ee1d2544a1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:17.469041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:17.469691 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:17.469972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:17.470246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:17.479946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:17.481416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:17.490784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1282/2521] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:18.101353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.101920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.102215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.128977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] MFA Rules not processed for user `0c88621a24264fbf9efbdc9db1abdb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:18.130305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:18.170157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:18.200239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1240/2522] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:50:18.204665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.214751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a1bf4-bd85-4ca5-8476-9b252105f2b4 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.219341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.219988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.220270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.220577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.221262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.223165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.249850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1283/2523] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:50:18 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:18.253794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.266955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470eec4-75bb-47d2-9473-2e3e5e831c35 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.324983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.325647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.325923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.326192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.326888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.328848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.347971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1241/2524] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:50:18 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:18.352178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.362788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ac39c-b2e9-4c5e-9c07-0b799b9df83f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.367803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.369039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.369321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.369594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.370588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.372163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.373562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:18.386728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1284/2525] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:50:18.388659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.398546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f39ab-e88b-4bbb-a1ca-ac994da2f593 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.403133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.403995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.404269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.404583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] PATH_INFO: `/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a/users/d7d605e1196a4b59866fdeaa5b21834a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.418742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e6c451e1774c4f90123abd1bf3aa3a, user_id=d7d605e1196a4b59866fdeaa5b21834a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.421449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.435882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1242/2526] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:18 2022] PUT /identity/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a/users/d7d605e1196a4b59866fdeaa5b21834a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:50:18.439740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.452119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5896d1-16c9-45c1-97bb-439a531817b6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.508626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.509793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.510069 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.510341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.511054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.512766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4114e0-7431-4833-bc01-bf4c9d605c81 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.522358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1285/2527] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:50:18.526439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:18.536563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78fb3af-cacb-463b-926d-c0d58db751e7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:18.541067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:18.541934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:18.542208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:18.542475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] PATH_INFO: `/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a/users/d7d605e1196a4b59866fdeaa5b21834a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:18.544453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66e6c451e1774c4f90123abd1bf3aa3a, user_id=d7d605e1196a4b59866fdeaa5b21834a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:18.547109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:18.557601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1243/2528] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:18 2022] PUT /identity/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a/users/d7d605e1196a4b59866fdeaa5b21834a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:19.191959 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef444-d371-4366-833a-814571784f1d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:19.192255 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef444-d371-4366-833a-814571784f1d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:19.192556 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfef444-d371-4366-833a-814571784f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:19.219541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-dcfef444-d371-4366-833a-814571784f1d None None] MFA Rules not processed for user `d7d605e1196a4b59866fdeaa5b21834a`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:19.220784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dcfef444-d371-4366-833a-814571784f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:19.290942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1286/2529] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:19 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:19.723194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:19.724942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:19.797560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d8ad8-035a-408d-b36d-0990a42e2368 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:19.802560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:19.803204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:19.803485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:19.803760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:19.805578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:19.817677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:19.819276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:19.830293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1244/2530] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:50:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:50:20.279354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfef444-d371-4366-833a-814571784f1d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:20.341501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfef444-d371-4366-833a-814571784f1d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:20.346252 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:20.347131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:20.347415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:20.347694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] PATH_INFO: `/v3/users/0c88621a24264fbf9efbdc9db1abdb5b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:20.348908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c88621a24264fbf9efbdc9db1abdb5b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:20.350867 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:20.378492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:20.405470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c88621a24264fbf9efbdc9db1abdb5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.406118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c88621a24264fbf9efbdc9db1abdb5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.406720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c88621a24264fbf9efbdc9db1abdb5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.413621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c88621a24264fbf9efbdc9db1abdb5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.421009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c88621a24264fbf9efbdc9db1abdb5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.438246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1287/2531] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:50:20 2022] DELETE /identity/v3/users/0c88621a24264fbf9efbdc9db1abdb5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:20.603782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:20.614775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e7504-6a95-430a-a843-5dcb4f7db517 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:20.631253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:20.673105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:20.673883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:20.674157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:20.674423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] PATH_INFO: `/v3/projects/7f6a5b6aab4d44fd8c06968e7eb2117d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:20.675573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f6a5b6aab4d44fd8c06968e7eb2117d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:20.677514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:20.721826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f6a5b6aab4d44fd8c06968e7eb2117d', 'request_id': 'req-71649395-a8a0-49a9-9fa1-834310fe74c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.727133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f6a5b6aab4d44fd8c06968e7eb2117d', 'request_id': 'req-71649395-a8a0-49a9-9fa1-834310fe74c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.734446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f6a5b6aab4d44fd8c06968e7eb2117d', 'request_id': 'req-71649395-a8a0-49a9-9fa1-834310fe74c7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.745313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] The token cache is being invalidate because project 7f6a5b6aab4d44fd8c06968e7eb2117d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:20.745610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.746205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.747692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1245/2532] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:50:20 2022] DELETE /identity/v3/projects/7f6a5b6aab4d44fd8c06968e7eb2117d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:20.751912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:20.753510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:20.767046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3aa007-54d1-4e8e-933d-49ed4ff35259 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:20.823551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:20.824451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:20.824730 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:20.825000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] PATH_INFO: `/v3/users/d7d605e1196a4b59866fdeaa5b21834a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:20.826160 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d605e1196a4b59866fdeaa5b21834a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:20.828062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:20.886931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d605e1196a4b59866fdeaa5b21834a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.887580 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d605e1196a4b59866fdeaa5b21834a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.888257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d605e1196a4b59866fdeaa5b21834a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.895544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d605e1196a4b59866fdeaa5b21834a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.903239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-eafb6299-5eed-4135-a135-7583c3807aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d605e1196a4b59866fdeaa5b21834a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:20.920114 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1288/2533] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:50:20 2022] DELETE /identity/v3/users/d7d605e1196a4b59866fdeaa5b21834a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:21.086595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.088863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:21.101540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71649395-a8a0-49a9-9fa1-834310fe74c7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.161664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.162651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.162936 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.163213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] PATH_INFO: `/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.164429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66e6c451e1774c4f90123abd1bf3aa3a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.166385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.212941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66e6c451e1774c4f90123abd1bf3aa3a', 'request_id': 'req-c72b471f-59d9-44e2-af4b-b73171da47a7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:21.214784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:21.219392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66e6c451e1774c4f90123abd1bf3aa3a', 'request_id': 'req-c72b471f-59d9-44e2-af4b-b73171da47a7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:21.226640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66e6c451e1774c4f90123abd1bf3aa3a', 'request_id': 'req-c72b471f-59d9-44e2-af4b-b73171da47a7'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:21.237746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] The token cache is being invalidate because project 66e6c451e1774c4f90123abd1bf3aa3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:50:21.238033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:21.238664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-c72b471f-59d9-44e2-af4b-b73171da47a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:50:21.240213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1246/2534] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:50:21 2022] DELETE /identity/v3/projects/66e6c451e1774c4f90123abd1bf3aa3a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:21.246003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.246411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.246791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.265432 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:21.266619 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:21.276102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:21.327618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1289/2535] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:21.335277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.335561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.335836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.353756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:21.354935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:21.362777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1247/2536] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:21.367125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.377667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc9d85-96eb-4e7d-a746-04937b15a685 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.382489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.383139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.383441 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.383699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.385107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:50:21.385417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.387022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.398373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1290/2537] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:50:21.402444 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.411668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af1ffc-6d8f-47f6-b930-b2351148acbe None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.416275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.416943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.417230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.417505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.418196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.420092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.445219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1248/2538] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:21.451192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.461284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a09b2-8da7-4e07-929d-dca102c563f3 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.516276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.516965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.517274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.517549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.518251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.520139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.539239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1291/2539] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:50:21.543251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.553365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322dd0c4-d6bc-490f-8e24-f72841043eb8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.558458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.559103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.559379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.559648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.560421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.562019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.573054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1249/2540] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:21.576719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.586542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498c0e9-4c26-49a7-8b7b-902adddf3784 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.591171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.591817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.592098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.592406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.593126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.594699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235cdc7-1822-4517-8e0a-cc02fd30dbff admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.604518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1292/2541] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:21.608282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.618798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4437b6-dfb3-4a10-a0f5-f8de4fb1e405 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.623572 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.624471 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.624751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.625022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] PATH_INFO: `/v3/projects/21b32ed73c74405cbba43e6e1528c543/users/60b89fe7bff8453980c644124b810f21/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.639814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21b32ed73c74405cbba43e6e1528c543, user_id=60b89fe7bff8453980c644124b810f21, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.642477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.655227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1250/2542] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:21 2022] PUT /identity/v3/projects/21b32ed73c74405cbba43e6e1528c543/users/60b89fe7bff8453980c644124b810f21/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:50:21.665731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.666028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.666307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.692557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] MFA Rules not processed for user `60b89fe7bff8453980c644124b810f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:21.693741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:21.755274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1293/2543] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:21.759555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.820224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc30c-4c07-49c1-92e3-94be2ced3fe5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.824835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.825480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.825762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.826039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.826724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.828685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.853776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1251/2544] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:21.857681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.869242 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f4a3f-2a21-4e8b-b97d-7da6001959f7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.877110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:21.926916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.927583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.927863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.928162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.928882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.931443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.951481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1294/2545] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:50:21 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:50:21.955524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.966114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6bb7c-78fe-429b-a9fe-2deac670c172 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:21.970735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:21.971388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:21.971672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:21.971948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:21.972717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:21.974341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:21.985319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1252/2546] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:21.989163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:21.999655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00518a32-0652-4f13-bcce-18945c416e88 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:22.004375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:22.005299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.005598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.005873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] PATH_INFO: `/v3/projects/35b34fab3a384977922ee0dae46bfb53/users/0a94d0b259ca48b3bc454631040b66e3/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.020057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b34fab3a384977922ee0dae46bfb53, user_id=0a94d0b259ca48b3bc454631040b66e3, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:22.022717 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:22.035594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1295/2547] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:21 2022] PUT /identity/v3/projects/35b34fab3a384977922ee0dae46bfb53/users/0a94d0b259ca48b3bc454631040b66e3/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:22.039438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:22.049997 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f976eca-fc15-4b68-856b-ee4870f80cf5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:22.096886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:22.109253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:22.109924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.110206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.110481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.111204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:22.112888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e0617-27d8-45b3-b997-30350df3a110 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:22.122785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1253/2548] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:50:22.126880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:50:22.137094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab70d878-bf14-44f9-b152-24b7ef432f3c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:50:22.141677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:50:22.142546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.142830 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.143107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] PATH_INFO: `/v3/projects/35b34fab3a384977922ee0dae46bfb53/users/0a94d0b259ca48b3bc454631040b66e3/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.145088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b34fab3a384977922ee0dae46bfb53, user_id=0a94d0b259ca48b3bc454631040b66e3, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:50:22.147695 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:50:22.158203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1296/2549] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:50:22 2022] PUT /identity/v3/projects/35b34fab3a384977922ee0dae46bfb53/users/0a94d0b259ca48b3bc454631040b66e3/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:50:22.168576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:50:22.168867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:50:22.169147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:50:22.194729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] MFA Rules not processed for user `0a94d0b259ca48b3bc454631040b66e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:50:22.195917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:50:22.262635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1254/2550] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:50:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:50:32.154598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=39,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:50:32.259732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=40 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:51:32.678304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:51:32.680110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:51:32.740633 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f78970-61eb-4463-b9aa-f8b01c568d8e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:51:32.744488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:51:32.745127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:51:32.745409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:51:32.745700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:51:32.747430 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:51:32.756664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:51:32.758082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:51:32.766439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1297/2551] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:51:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:51:42.762947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=26 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:13.404974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:13.406365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:13.472027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0504d03-5c21-47f3-8a73-af0818850990 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:13.475537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:13.476165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:13.476475 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:13.476746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:13.478449 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:13.546334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:13.547769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:13.556632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1255/2552] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:52:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:52:13.800275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:13.809596 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084e7f-8af1-4a5e-8beb-b3949daff865 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:13.813197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:13.813836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:13.814118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:13.814392 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:13.823246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:13.824741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245fedd-f811-495f-8caf-5014b21e879d service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:13.834261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1298/2553] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:52:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:52:14.141740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:14.210766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25db095-3ac2-421c-a076-6bcd8b1a486f service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:14.215958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:14.216865 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:14.217159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:14.217434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] PATH_INFO: `/v3/users/60b89fe7bff8453980c644124b810f21` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:14.218601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60b89fe7bff8453980c644124b810f21)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:14.220554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:14.256484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:14.267007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60b89fe7bff8453980c644124b810f21'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.267654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60b89fe7bff8453980c644124b810f21'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.268236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60b89fe7bff8453980c644124b810f21'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.278412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60b89fe7bff8453980c644124b810f21'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.291158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60b89fe7bff8453980c644124b810f21'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.303338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1256/2554] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:52:14 2022] DELETE /identity/v3/users/60b89fe7bff8453980c644124b810f21 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:14.307871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:14.308173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:14.308607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:14.326222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:52:14.327404 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:14.385048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1299/2555] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:52:14.395838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:14.397334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:14.454446 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5fa5c-4048-4b23-8b5e-6d94c24e57a7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:14.458111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:14.458754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:14.459034 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:14.459313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:14.468288 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:14.469782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:14.479122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1257/2556] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:52:14.747882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:14.749474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:14.761284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7867f5-a6c9-4b49-a320-c94224f3eeb8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:14.765939 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:14.766832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:14.767117 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:14.767386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] PATH_INFO: `/v3/projects/21b32ed73c74405cbba43e6e1528c543` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:14.768627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=21b32ed73c74405cbba43e6e1528c543)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:14.770536 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:14.823256 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21b32ed73c74405cbba43e6e1528c543', 'request_id': 'req-7d5a9627-324f-4916-9f85-138f2b67ed0b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.828732 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21b32ed73c74405cbba43e6e1528c543', 'request_id': 'req-7d5a9627-324f-4916-9f85-138f2b67ed0b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.835747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21b32ed73c74405cbba43e6e1528c543', 'request_id': 'req-7d5a9627-324f-4916-9f85-138f2b67ed0b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.846763 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] The token cache is being invalidate because project 21b32ed73c74405cbba43e6e1528c543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:52:14.847048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.847690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.849281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1300/2557] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:52:14 2022] DELETE /identity/v3/projects/21b32ed73c74405cbba43e6e1528c543 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:14.853424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:14.863567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce816d0a-9807-4400-98df-e1c525984070 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:14.925571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:14.926368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:14.926649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:14.926923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] PATH_INFO: `/v3/users/0a94d0b259ca48b3bc454631040b66e3` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:14.928087 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a94d0b259ca48b3bc454631040b66e3)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:14.930069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:14.974388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a94d0b259ca48b3bc454631040b66e3'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.975040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a94d0b259ca48b3bc454631040b66e3'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.975640 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a94d0b259ca48b3bc454631040b66e3'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.983023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a94d0b259ca48b3bc454631040b66e3'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:14.984195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:14.997636 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b7dafab8-e214-44f8-b192-0feb66727133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a94d0b259ca48b3bc454631040b66e3'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.010948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1258/2558] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:52:14 2022] DELETE /identity/v3/users/0a94d0b259ca48b3bc454631040b66e3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:15.270532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.272027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:15.285754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a9627-324f-4916-9f85-138f2b67ed0b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.348059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.348973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.349238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.349511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] PATH_INFO: `/v3/projects/35b34fab3a384977922ee0dae46bfb53` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.350676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b34fab3a384977922ee0dae46bfb53)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.352623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.364837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:15.399549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b34fab3a384977922ee0dae46bfb53', 'request_id': 'req-4e3aa601-acfc-4996-b7c7-f96126dcdab1'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.405380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b34fab3a384977922ee0dae46bfb53', 'request_id': 'req-4e3aa601-acfc-4996-b7c7-f96126dcdab1'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.412202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b34fab3a384977922ee0dae46bfb53', 'request_id': 'req-4e3aa601-acfc-4996-b7c7-f96126dcdab1'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.423714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] The token cache is being invalidate because project 35b34fab3a384977922ee0dae46bfb53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:52:15.423999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.424652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4e3aa601-acfc-4996-b7c7-f96126dcdab1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:52:15.426213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1301/2559] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:52:15 2022] DELETE /identity/v3/projects/35b34fab3a384977922ee0dae46bfb53 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:15.431106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a610c-c4d8-4301-b381-12172310227c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.431397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a610c-c4d8-4301-b381-12172310227c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.431672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a610c-c4d8-4301-b381-12172310227c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.449655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-739a610c-c4d8-4301-b381-12172310227c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:52:15.450770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-739a610c-c4d8-4301-b381-12172310227c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:15.508916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1259/2560] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:52:15.518202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.518496 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.518776 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.537184 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:52:15.538384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:15.546183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1302/2561] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:52:15.550219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a610c-c4d8-4301-b381-12172310227c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.559657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a610c-c4d8-4301-b381-12172310227c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.564432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.565079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.565362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.565635 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.566351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:52:15.566658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.568265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.591736 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1260/2562] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:52:15.595803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.606589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd3d7f-5340-4a32-b546-2f515b75d89d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.612131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.613066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.613475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.613905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.614603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.616532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.646615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1303/2563] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:52:15.652411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.663290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fca25c-2c7c-4750-ad62-868949c3a0b6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.718234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.718899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.719188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.719467 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.720150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.722131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.741224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1261/2564] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:52:15.745403 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.755644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc74796-7de6-49f8-a7e2-630ee40983cd admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.760471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.761660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.761941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.762239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.762996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.764634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.774985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1304/2565] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:52:15.779338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.789192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d094a0a-8f3e-4c6c-a033-ecc17599c009 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.794435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.795086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.795370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.795649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.796438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.798082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3ed06-cc85-468b-b11f-0eb91f9d3955 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.809859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1262/2566] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:52:15.813833 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:15.824230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5af339-57ea-46e3-9c25-b4eccfd37eba admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:15.829006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:15.829874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.830162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.830439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] PATH_INFO: `/v3/projects/3a28dcb488d74bbda398d95f623ad1b5/users/a83083471ae64281bba8f027e231ad3a/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.845325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a28dcb488d74bbda398d95f623ad1b5, user_id=a83083471ae64281bba8f027e231ad3a, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:15.847985 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:15.860956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1305/2567] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:52:15 2022] PUT /identity/v3/projects/3a28dcb488d74bbda398d95f623ad1b5/users/a83083471ae64281bba8f027e231ad3a/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:15.871583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:15.871890 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:15.872168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:15.898312 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] MFA Rules not processed for user `a83083471ae64281bba8f027e231ad3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:52:15.899505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:15.927338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:15.966742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1263/2568] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:52:15.971330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.031892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7834f450-cdf9-458e-8ae7-99c443b0bea0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.036557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.037214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.037497 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.037775 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.038468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.040400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.068655 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1306/2569] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:52:15 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:52:16.068869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.080178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7964746-469b-4eec-88a0-8113c727fb6d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.134338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.135003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.135284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.135562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.136880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.138808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.158423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1264/2570] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:52:16 2022] POST /identity/v3/users => generated 303 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:52:16.162397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.172527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da2ece-8c4b-49a9-84de-a7b0838816c0 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.177023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.177668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.177947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.178219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.178943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.180560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.191307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1307/2571] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:52:16.195340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.205821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51157ed8-756b-403c-9a1a-3b181ffa22c0 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.210415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.211281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.211561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.211835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] PATH_INFO: `/v3/projects/a5fec20301c544ccb6bb7a80b9abf536/users/8a8782862a564a138ac5b5afd98647c6/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.226266 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fec20301c544ccb6bb7a80b9abf536, user_id=8a8782862a564a138ac5b5afd98647c6, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.228982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.246083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1265/2572] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:52:16 2022] PUT /identity/v3/projects/a5fec20301c544ccb6bb7a80b9abf536/users/8a8782862a564a138ac5b5afd98647c6/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:16.250138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.259805 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d0e409-9a1c-4f1c-9aeb-51d4cd232b7f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.267424 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:16.315973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.316664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.316944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.317217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.317930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.319518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148969b6-7e96-4528-ab72-083a2cc0f019 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.329033 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1308/2573] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:52:16.333013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:16.342977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50bc7c-727b-4b33-852e-7673c5de3d09 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:16.347472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:16.348359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.348642 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.348916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] PATH_INFO: `/v3/projects/a5fec20301c544ccb6bb7a80b9abf536/users/8a8782862a564a138ac5b5afd98647c6/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.350832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fec20301c544ccb6bb7a80b9abf536, user_id=8a8782862a564a138ac5b5afd98647c6, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:16.353477 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:16.364104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1266/2574] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:52:16 2022] PUT /identity/v3/projects/a5fec20301c544ccb6bb7a80b9abf536/users/8a8782862a564a138ac5b5afd98647c6/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:52:16.374887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:16.375174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:16.375449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:16.401488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] MFA Rules not processed for user `8a8782862a564a138ac5b5afd98647c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:52:16.402723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:16.469782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1309/2575] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:52:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:52:17.487474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:52:17.489251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:52:17.544750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2de6c3-506b-498b-b47c-bb319eea766e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:52:17.548231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:52:17.548884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:52:17.549162 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:52:17.549433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:52:17.558685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:52:17.560114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:52:17.569345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1267/2576] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:52:17 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:52:26.466919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=58 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:52:27.565269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=84,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:53:31.031571 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:53:31.033051 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:31.130810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9a193-6f74-4cb0-b8c0-9e659ba26f19 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:53:31.135784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:53:31.136470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:53:31.136749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:53:31.137019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:53:31.139345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:53:31.151007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:53:31.152708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:53:31.162638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1310/2577] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:53:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:53:41.158415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=45 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:14.170379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:14.171762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:14.221986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01597461-5c41-4b01-9a38-49b80143742b service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:14.225582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:14.226214 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:14.226495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:14.226767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:14.281605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:14.282993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:14.291705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1268/2578] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:54:14.760860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:14.846517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8438a2e-d298-4cef-8209-223e2459f6d6 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:14.851832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:14.852878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:14.853158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:14.853431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] PATH_INFO: `/v3/users/a83083471ae64281bba8f027e231ad3a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:14.854657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=a83083471ae64281bba8f027e231ad3a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:14.857489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:14.868523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:14.877476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b03d37-ebfd-415d-b47e-03c0946baf82 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:14.881041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:14.881674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:14.881951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:14.882218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:14.883963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:14.906423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a83083471ae64281bba8f027e231ad3a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:14.907064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a83083471ae64281bba8f027e231ad3a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:14.907726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a83083471ae64281bba8f027e231ad3a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:14.916268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a83083471ae64281bba8f027e231ad3a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:14.924015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-70651d95-d08a-4e24-93d4-367162e724ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a83083471ae64281bba8f027e231ad3a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:14.941964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1311/2579] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:54:14 2022] DELETE /identity/v3/users/a83083471ae64281bba8f027e231ad3a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:14.946226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:14.946578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:14.946862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:14.955903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:14.957394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:14.975967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1269/2580] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:54:14.976866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:14.978145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:15.020010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:15.039018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1312/2581] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:15.049801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:15.051161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:15.096208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:15.108591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a03522-7444-481f-9424-898d0d8a7cae service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:15.154958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:15.155605 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:15.155887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:15.156161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:15.164986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:15.166432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:15.176432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1270/2582] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:15.456760 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:15.458262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:15.470806 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3492-9dbd-456e-a409-9825ec8dc7ae None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:15.475446 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:15.476331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:15.476609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:15.476878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] PATH_INFO: `/v3/projects/3a28dcb488d74bbda398d95f623ad1b5` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:15.478053 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a28dcb488d74bbda398d95f623ad1b5)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:15.479964 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:15.527382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a28dcb488d74bbda398d95f623ad1b5', 'request_id': 'req-3ab2c06b-62db-40de-82cc-cad637d5991b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.533364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a28dcb488d74bbda398d95f623ad1b5', 'request_id': 'req-3ab2c06b-62db-40de-82cc-cad637d5991b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.540485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a28dcb488d74bbda398d95f623ad1b5', 'request_id': 'req-3ab2c06b-62db-40de-82cc-cad637d5991b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.551251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] The token cache is being invalidate because project 3a28dcb488d74bbda398d95f623ad1b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:54:15.551535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.552155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.553724 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1313/2583] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:54:15 2022] DELETE /identity/v3/projects/3a28dcb488d74bbda398d95f623ad1b5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:15.557600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:15.567532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605cf9-f873-4142-8ef9-5766c04be89e service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:15.622703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:15.623563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:15.623845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:15.624120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] PATH_INFO: `/v3/users/8a8782862a564a138ac5b5afd98647c6` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:15.625330 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8782862a564a138ac5b5afd98647c6)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:15.627272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:15.676175 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8782862a564a138ac5b5afd98647c6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.676857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8782862a564a138ac5b5afd98647c6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.677463 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8782862a564a138ac5b5afd98647c6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.684279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8782862a564a138ac5b5afd98647c6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.695856 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-79924040-acc2-475c-b650-ce98efe11cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8782862a564a138ac5b5afd98647c6'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:15.708974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1271/2584] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:54:15 2022] DELETE /identity/v3/users/8a8782862a564a138ac5b5afd98647c6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:15.978676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:15.980174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:15.993947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2c06b-62db-40de-82cc-cad637d5991b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.051034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.051893 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.052169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.052477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] PATH_INFO: `/v3/projects/a5fec20301c544ccb6bb7a80b9abf536` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.053628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fec20301c544ccb6bb7a80b9abf536)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.055525 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.104452 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fec20301c544ccb6bb7a80b9abf536', 'request_id': 'req-f26406d3-db8f-4eb7-9e42-4f94288035e8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:16.110372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fec20301c544ccb6bb7a80b9abf536', 'request_id': 'req-f26406d3-db8f-4eb7-9e42-4f94288035e8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:16.117761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fec20301c544ccb6bb7a80b9abf536', 'request_id': 'req-f26406d3-db8f-4eb7-9e42-4f94288035e8'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:16.126640 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:16.131238 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] The token cache is being invalidate because project a5fec20301c544ccb6bb7a80b9abf536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:54:16.131734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:16.132793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f26406d3-db8f-4eb7-9e42-4f94288035e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:16.134573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1314/2585] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:54:15 2022] DELETE /identity/v3/projects/a5fec20301c544ccb6bb7a80b9abf536 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:16.149123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.149409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.149691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.166866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:16.168056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:16.181144 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:16.228454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1272/2586] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:16.284066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.284413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.284693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.303289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:16.304506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:16.312947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1315/2587] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:16.317124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.326957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9aadb-71c4-42d6-a31c-8388f9749f23 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.331429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.332074 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.332389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.332666 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.333385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:54:16.333693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.335302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.346616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1273/2588] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:54:16.350766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.360490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cadfd23-3c00-466c-9dc7-fbe98eaa4761 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.364971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.365613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.365889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.366155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.366837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.369031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.394368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1316/2589] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:54:16.398427 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.408410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7c592-5ff1-44e3-bed2-8c5fd07cae6c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.465009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.465671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.465957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.466235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.466922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.468900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.488774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1274/2590] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:54:16.493091 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.503831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7c327-0271-4d51-b38c-e734b6b36f78 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.508401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.509050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.509330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.509601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.510322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.511906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.522143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1317/2591] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:54:16.525917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.536229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02faebd-6c31-4b5b-8a83-5d2c6736e5ef admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.540889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.541537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.541822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.542101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.542825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.545082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782089e-0c4e-4751-8b9d-beb3fd0ec506 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.554795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1275/2592] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:54:16.558541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.568480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06dc12-7b61-4ea7-9f8d-4822fd00a2be admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.573174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.574042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.574327 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.574603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] PATH_INFO: `/v3/projects/43d9f743f041467ab28d2152518f223b/users/688c6357e636485b871e388c06a920da/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.589582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=43d9f743f041467ab28d2152518f223b, user_id=688c6357e636485b871e388c06a920da, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.593666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.606736 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1318/2593] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:54:16 2022] PUT /identity/v3/projects/43d9f743f041467ab28d2152518f223b/users/688c6357e636485b871e388c06a920da/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:16.616772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.617009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.617293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.643485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] MFA Rules not processed for user `688c6357e636485b871e388c06a920da`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:16.644695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:16.707938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1276/2594] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:54:16.712293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.772645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923a357-9604-4849-8656-af020ab2b0aa admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.777269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.778472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.778753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.779020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.779703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.781630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.806526 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1319/2595] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:54:16.810616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.821041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760fe1a-2910-4717-9a5d-df15cea55dcc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.832227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:16.878042 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.878654 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.878912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.879183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.879872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.881824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.902143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1277/2596] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:54:16 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:54:16.906638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.916870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e895d1-0870-421e-a03b-ed8e65650c51 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.921471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.922126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.922409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.922712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.923439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.925073 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.936471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1320/2597] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:54:16.940375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:16.951479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ebd3c-8397-4b84-97b1-2ec74aa001a7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:16.956027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:16.956950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:16.957212 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:16.957483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] PATH_INFO: `/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce/users/b089e2fc0cf14b6f90fcf7fbe011c71a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:16.972561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ed50eeed6b4526b17c89e55d09b6ce, user_id=b089e2fc0cf14b6f90fcf7fbe011c71a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:16.975213 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:16.987728 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1278/2598] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:54:16 2022] PUT /identity/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce/users/b089e2fc0cf14b6f90fcf7fbe011c71a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:16.991584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:17.001568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8e2db-bf80-4d92-9d37-a88ae5326d30 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:17.056731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:17.057402 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:17.057685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:17.057962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:17.058685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:17.060368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44dcb4-d074-4fa7-bdf8-63064c08c228 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:17.070093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1321/2599] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:54:17.074514 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:17.085591 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295570e-4b48-4351-a6e9-35e72be81811 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:17.090183 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:17.091694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:17.091975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:17.092250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] PATH_INFO: `/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce/users/b089e2fc0cf14b6f90fcf7fbe011c71a/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:17.094260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ed50eeed6b4526b17c89e55d09b6ce, user_id=b089e2fc0cf14b6f90fcf7fbe011c71a, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:17.096954 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:17.107146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1279/2600] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:54:17 2022] PUT /identity/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce/users/b089e2fc0cf14b6f90fcf7fbe011c71a/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:17.117126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:17.117425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:17.117951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:17.130101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:17.144823 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] MFA Rules not processed for user `b089e2fc0cf14b6f90fcf7fbe011c71a`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:17.146022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:17.217010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1322/2601] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:17.748015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:17.749720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:17.805243 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68a81b-7a8a-4b01-aa92-536a90c180c1 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:17.809059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:17.809710 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:17.809987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:17.810257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:17.818987 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:17.820461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea3a5a0-04bb-472c-8040-9fb1bddc26a5 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:17.830195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1280/2602] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:18.174014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:18.236056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61339a1b-26fa-4f01-9bbc-95d6e9fef820 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:18.241272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:18.242150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:18.242700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:18.242951 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] PATH_INFO: `/v3/users/688c6357e636485b871e388c06a920da` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:18.244146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=688c6357e636485b871e388c06a920da)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:18.246219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:18.291715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688c6357e636485b871e388c06a920da'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.292405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688c6357e636485b871e388c06a920da'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.293040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688c6357e636485b871e388c06a920da'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.300628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688c6357e636485b871e388c06a920da'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.307405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688c6357e636485b871e388c06a920da'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.324650 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1323/2603] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:54:18 2022] DELETE /identity/v3/users/688c6357e636485b871e388c06a920da => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:18.329667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:18.329960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:18.330240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:18.347498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:18.348737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:18.410005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1281/2604] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:18.420251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:18.421697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:18.435715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:18.480378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57cfd42-132b-4ca4-a56a-35f70db3edc4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:18.484551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:18.485200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:18.485484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:18.485757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:18.501078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:18.502442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:18.512931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1324/2605] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:18.790612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:18.792060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:18.804735 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699421c0-2f0e-4207-84ff-cdbf739402c0 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:18.809917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:18.810771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:18.811051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:18.811327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] PATH_INFO: `/v3/projects/43d9f743f041467ab28d2152518f223b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:18.812559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43d9f743f041467ab28d2152518f223b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:18.814479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:18.821136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:18.861938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43d9f743f041467ab28d2152518f223b', 'request_id': 'req-3f849735-cd11-444f-abc5-f07b6c000cc5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.867900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43d9f743f041467ab28d2152518f223b', 'request_id': 'req-3f849735-cd11-444f-abc5-f07b6c000cc5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.875468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43d9f743f041467ab28d2152518f223b', 'request_id': 'req-3f849735-cd11-444f-abc5-f07b6c000cc5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.886308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] The token cache is being invalidate because project 43d9f743f041467ab28d2152518f223b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:54:18.886620 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.887849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:18.889483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1282/2606] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:54:18 2022] DELETE /identity/v3/projects/43d9f743f041467ab28d2152518f223b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:18.893399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:18.903245 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aedc9c-24fc-4b42-8938-9d2c5a475d61 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:18.958011 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:18.958875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:18.959162 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:18.959434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] PATH_INFO: `/v3/users/b089e2fc0cf14b6f90fcf7fbe011c71a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:18.960636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b089e2fc0cf14b6f90fcf7fbe011c71a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:18.962550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.006276 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b089e2fc0cf14b6f90fcf7fbe011c71a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.006908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b089e2fc0cf14b6f90fcf7fbe011c71a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.007506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b089e2fc0cf14b6f90fcf7fbe011c71a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.014942 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b089e2fc0cf14b6f90fcf7fbe011c71a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.021885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-2b51daef-8371-47a0-bc36-763e846aac16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b089e2fc0cf14b6f90fcf7fbe011c71a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.038733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1325/2607] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:54:18 2022] DELETE /identity/v3/users/b089e2fc0cf14b6f90fcf7fbe011c71a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:19.302101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.303574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:19.317669 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f849735-cd11-444f-abc5-f07b6c000cc5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.376091 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.376999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.377272 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.377546 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] PATH_INFO: `/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.378731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ed50eeed6b4526b17c89e55d09b6ce)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.380705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.430999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ed50eeed6b4526b17c89e55d09b6ce', 'request_id': 'req-b9a15ad0-654e-454b-9663-cb3d6e3efd11'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.436428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ed50eeed6b4526b17c89e55d09b6ce', 'request_id': 'req-b9a15ad0-654e-454b-9663-cb3d6e3efd11'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.444136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ed50eeed6b4526b17c89e55d09b6ce', 'request_id': 'req-b9a15ad0-654e-454b-9663-cb3d6e3efd11'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.456507 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] The token cache is being invalidate because project 08ed50eeed6b4526b17c89e55d09b6ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:54:19.456794 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.457420 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-b9a15ad0-654e-454b-9663-cb3d6e3efd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:54:19.459188 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1283/2608] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:54:19 2022] DELETE /identity/v3/projects/08ed50eeed6b4526b17c89e55d09b6ce => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:19.463938 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.464464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.464560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.511821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:19.523406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:19.525107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:19.584389 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1326/2609] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:19.592256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.592641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.592989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.612159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:19.613487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:19.622812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1284/2610] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:54:19.627174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.637134 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc908cc6-debc-4022-bf12-89994b393150 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.643391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.644163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.644549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.644894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.645786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:54:19.646166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.648097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.664890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1327/2611] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:54:19.666113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.675877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c35b19-0015-40d8-a696-e74ead3969bc None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.680741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.682025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.682338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.682617 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.683304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.685848 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.701600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:19.710999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.713531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:19.733366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1285/2612] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:54:19 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:54:19.734565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.794935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9e77bf-100d-4925-a23d-d23d98991dcc admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.800876 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0ac6c-1fb2-4256-a177-2f20d23d4376 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.805544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.806237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.806521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.806797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.807481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.809978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.833673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1286/2613] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:54:19 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:54:19.834711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.844918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f0a61d-9b9e-4d62-b4ed-a1bb28d5df71 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.849844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.850571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.850859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.851136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.851861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.853604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.862781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.863436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.863719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.863992 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.864394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1287/2614] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:54:19.865930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:19.868408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.878824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26708-340e-497d-bec9-42c67575ad6a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.879347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.881021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.883589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.884280 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.884600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.884877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.885625 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.887752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ceb77-48f2-4d46-949d-250f7c0806c8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.891972 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1328/2615] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:19.900337 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1288/2616] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:54:19.902418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:19.913365 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82628a00-95fd-4c10-a517-879c341aa99d service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:19.918119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:19.919067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.919347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.919616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] PATH_INFO: `/v3/projects/9fb11d56c6f44dc1869eaf8e9b02cf9a/users/31679c3e3227466284955c72bd8c741e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.933838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:19.938078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb11d56c6f44dc1869eaf8e9b02cf9a, user_id=31679c3e3227466284955c72bd8c741e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:19.940743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:19.953277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1329/2617] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:54:19 2022] PUT /identity/v3/projects/9fb11d56c6f44dc1869eaf8e9b02cf9a/users/31679c3e3227466284955c72bd8c741e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:54:19.963540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:19.963841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:19.964147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:19.993483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] MFA Rules not processed for user `31679c3e3227466284955c72bd8c741e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:54:19.994699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:20.062563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1289/2618] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:54:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:54:20.074270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:54:20.077035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:54:20.160540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa13cb-e430-44d2-a7ac-fa55364aad65 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:54:20.164187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847292d322f6483ba5ff02294ad409dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:54:20.164860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:54:20.165142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:54:20.165416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:54:20.173613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:54:20.175031 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf8493a-7c7a-452a-ab44-275fc92f4a03 service glance] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:54:20.183696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1330/2619] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:54:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:54:30.059658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=76,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:54:30.179740 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:20.265748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:20.327076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd330f4-82b3-4d9f-a49f-11f07367d4be None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:20.331773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:20.332659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:20.332939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:20.333208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] PATH_INFO: `/v3/users/31679c3e3227466284955c72bd8c741e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:20.334371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31679c3e3227466284955c72bd8c741e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:20.336287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:20.382793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31679c3e3227466284955c72bd8c741e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:20.383478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31679c3e3227466284955c72bd8c741e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:20.384204 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31679c3e3227466284955c72bd8c741e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:20.392118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31679c3e3227466284955c72bd8c741e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:20.405492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31679c3e3227466284955c72bd8c741e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:20.419574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1290/2620] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:20 2022] DELETE /identity/v3/users/31679c3e3227466284955c72bd8c741e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:20.424385 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:20.424917 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:20.425022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:20.447230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:20.448725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.465663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:20.467268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.532587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1331/2621] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:55:20.533838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:20.535137 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.535342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1ef44-5ef1-42a7-80a9-2ddb074e98b8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:20.541535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:20.542381 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:20.542772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:20.543725 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:20.547111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.549171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf91c25-c503-4125-b7f3-e05f8d21e71d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:20.556957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:20.558302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:20.558613 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:20.559135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:20.561379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.594771 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:20.626802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:20.628247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:20.638612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1332/2622] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:20 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:20.641142 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:20.642483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:20.651007 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:20.652554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:20.674102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1291/2623] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:55:20.708007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c4f4b-3714-4e22-a8ae-2ab79122d637 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:20.712879 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:20.713522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:20.713801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:20.714068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:20.723752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:20.725273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:20.736354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1333/2624] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:21.166785 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.169391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:21.197464 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2085-74a4-43fb-b415-7b03e9ac923e service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.200955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.205832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.207164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.207595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.208454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] PATH_INFO: `/v3/projects/9fb11d56c6f44dc1869eaf8e9b02cf9a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.209905 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb11d56c6f44dc1869eaf8e9b02cf9a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.212729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.214582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b341-eff0-406a-ad97-7099669cfaee service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.226022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.226966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.227409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.227857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.230108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:21.305882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:21.315171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb11d56c6f44dc1869eaf8e9b02cf9a', 'request_id': 'req-51bb59e6-b799-4904-b858-eac9cdb944ad'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:21.322291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb11d56c6f44dc1869eaf8e9b02cf9a', 'request_id': 'req-51bb59e6-b799-4904-b858-eac9cdb944ad'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:21.333436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb11d56c6f44dc1869eaf8e9b02cf9a', 'request_id': 'req-51bb59e6-b799-4904-b858-eac9cdb944ad'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:21.352019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] The token cache is being invalidate because project 9fb11d56c6f44dc1869eaf8e9b02cf9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:21.352519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:21.353255 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-51bb59e6-b799-4904-b858-eac9cdb944ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:21.355718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1292/2625] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:21 2022] DELETE /identity/v3/projects/9fb11d56c6f44dc1869eaf8e9b02cf9a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:21.362164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.364067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e25b54-e0dc-409c-b0d0-f847e3c99bb2 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.364849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.365231 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.365569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.378837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1334/2626] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:55:21.389292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:21.391638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:21.456268 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1293/2627] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:55:21.464698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.465170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.465289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.483565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:21.484808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:21.493251 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1335/2628] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:55:21.497716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.507663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a964d1-a269-4c79-b595-d49805406f24 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.512831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.513492 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.513773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.514048 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.514767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:21.515070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.516706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.532333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1294/2629] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:55:21.533463 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.546815 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03330c-c83f-448e-a756-b6896d181310 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.554784 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.555451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.555774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.556036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.556749 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.558881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.596789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1336/2630] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:21.598529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.608612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e53481-5b4a-4626-952d-14cedd38bf04 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.622156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:21.673688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.674622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.674909 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.675151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.675896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.677999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.704778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1295/2631] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:21.706615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.717431 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae623008-43a1-4e5b-92a6-a69fc8bf6979 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.722121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.722789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.723101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.723372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.724097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.725774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.739661 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1337/2632] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:55:21.740883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.752434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a29ba-ee89-4460-b579-d7f773ad634d admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.765653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.767518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.770838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.771429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.772488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.776748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d96e6-6907-4670-8d78-f2de57711c15 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.788889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1296/2633] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:21.792888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:21.803202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4affe-c343-4d15-92c1-877832be83f4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:21.807865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:21.808967 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.809699 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.809971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] PATH_INFO: `/v3/projects/22d1c78d0103455bba61cd6b0a4a627b/users/80b95d65bbd34435884b33b4cb855f93/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.824408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d1c78d0103455bba61cd6b0a4a627b, user_id=80b95d65bbd34435884b33b4cb855f93, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:21.827697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:21.839958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1338/2634] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:21 2022] PUT /identity/v3/projects/22d1c78d0103455bba61cd6b0a4a627b/users/80b95d65bbd34435884b33b4cb855f93/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:55:21.849872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:21.850160 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:21.850438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:21.876855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] MFA Rules not processed for user `80b95d65bbd34435884b33b4cb855f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:21.878027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:21.944273 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1297/2635] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:21.948819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.010962 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9161e-5ea9-493f-b403-8afb9c1cca02 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.016220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.016907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.017189 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.017465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.018148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.020061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.045020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1339/2636] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:21 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:22.048900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.060163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea4f77-4a54-47c6-9e2d-f2cd5c0bd82d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.116375 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.117026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.117304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.117831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.118528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.120474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.140917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1298/2637] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:22 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:22.144976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.155576 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f54c5f-d618-445f-bf2d-671254ac2808 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.160477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.161139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.161422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.161700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.162458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.164224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.176808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1340/2638] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:22.180971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.191700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19980bcd-240f-4bb2-835c-29b96f7e31c8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.196384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.197283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.197570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.197845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] PATH_INFO: `/v3/projects/7467845b8c774292a85b4ac55eab0993/users/bb0db385bc574121b4fa5748c9099b3e/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.209379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:22.214570 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7467845b8c774292a85b4ac55eab0993, user_id=bb0db385bc574121b4fa5748c9099b3e, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.218432 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.234063 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1299/2639] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:22 2022] PUT /identity/v3/projects/7467845b8c774292a85b4ac55eab0993/users/bb0db385bc574121b4fa5748c9099b3e/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:22.237733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.248501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfd460-b37f-4960-82a7-9a2267c835bf admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.282368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:22.306801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.307471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.307750 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.308022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.308871 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.311061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b43162-4859-4117-b4d7-13053b0c01bc admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.320618 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1341/2640] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:22.324628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.334986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8838e78-0e38-4b02-9a41-11c569f8cdb5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.339750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.340808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.341086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.341356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] PATH_INFO: `/v3/projects/7467845b8c774292a85b4ac55eab0993/users/bb0db385bc574121b4fa5748c9099b3e/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.343301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7467845b8c774292a85b4ac55eab0993, user_id=bb0db385bc574121b4fa5748c9099b3e, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.345973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.356840 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1300/2641] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:22 2022] PUT /identity/v3/projects/7467845b8c774292a85b4ac55eab0993/users/bb0db385bc574121b4fa5748c9099b3e/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:22.366824 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.367108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.367384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.393683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] MFA Rules not processed for user `bb0db385bc574121b4fa5748c9099b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:22.394876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:22.460087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1342/2642] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:22.471238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:22.472949 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:22.528681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0dec9-afca-4037-b3d6-e2efd8014558 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:22.532258 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:22.533032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:22.533313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:22.533587 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:22.587357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:22.588773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:22.598105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1301/2643] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:23.357029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:23.358445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:23.417144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c69141-9905-48f2-8cdf-97ba7fb5a37f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:23.420629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:23.421265 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:23.421540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:23.421810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:23.423551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:23.492729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:23.494206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:23.500601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:23.506126 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1343/2644] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:25.640203 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:25.641722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:25.661136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25748671-8c49-450f-9077-26c6b0c3f3a3 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:25.667547 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:25.669246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:25.669567 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:25.669999 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:25.672523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:25.689147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:25.690952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe8941-1431-4adb-9a5f-97a9caa8f414 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:25.708501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1302/2645] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:26.182671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:26.265574 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d198e7-6e4c-42b5-8ab5-e7d8d9b400ac service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:26.271733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:26.272666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:26.272947 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:26.273221 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] PATH_INFO: `/v3/users/80b95d65bbd34435884b33b4cb855f93` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:26.274398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b95d65bbd34435884b33b4cb855f93)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:26.276323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:26.336542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b95d65bbd34435884b33b4cb855f93'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.337236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b95d65bbd34435884b33b4cb855f93'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.337858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b95d65bbd34435884b33b4cb855f93'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.347605 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b95d65bbd34435884b33b4cb855f93'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.357006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b95d65bbd34435884b33b4cb855f93'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.382442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1344/2646] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:26 2022] DELETE /identity/v3/users/80b95d65bbd34435884b33b4cb855f93 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:26.389232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:26.389606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:26.389866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:26.409628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:26.410901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:26.475085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1303/2647] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:26.484658 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:26.486072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:26.548464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89650772-10b5-4c36-92f6-44096974ee50 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:26.552100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:26.552787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:26.553071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:26.553352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:26.562895 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:26.564440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:26.573819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1345/2648] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:26.854317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:26.855825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:26.868577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77117cc-03a5-490a-8775-464d2d4d4a82 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:26.875356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:26.876661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:26.877015 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:26.877948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] PATH_INFO: `/v3/projects/22d1c78d0103455bba61cd6b0a4a627b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:26.879417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d1c78d0103455bba61cd6b0a4a627b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:26.881899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:26.895421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:26.933663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d1c78d0103455bba61cd6b0a4a627b', 'request_id': 'req-10e0208c-8f45-4f00-a8df-00f01e1ad58c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.939607 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d1c78d0103455bba61cd6b0a4a627b', 'request_id': 'req-10e0208c-8f45-4f00-a8df-00f01e1ad58c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.950224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d1c78d0103455bba61cd6b0a4a627b', 'request_id': 'req-10e0208c-8f45-4f00-a8df-00f01e1ad58c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.964248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] The token cache is being invalidate because project 22d1c78d0103455bba61cd6b0a4a627b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:26.964573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.965925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:26.967585 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1304/2649] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:26 2022] DELETE /identity/v3/projects/22d1c78d0103455bba61cd6b0a4a627b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:26.971907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:26.983119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9f8cc-3d2d-4be8-968f-3355292b0380 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:27.004015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:27.057937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:27.058887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.059210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.059485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] PATH_INFO: `/v3/users/bb0db385bc574121b4fa5748c9099b3e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.060782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb0db385bc574121b4fa5748c9099b3e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:27.062702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:27.112240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb0db385bc574121b4fa5748c9099b3e'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.112909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb0db385bc574121b4fa5748c9099b3e'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.113510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb0db385bc574121b4fa5748c9099b3e'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.121204 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb0db385bc574121b4fa5748c9099b3e'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.128643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-ab6d78fe-2429-43af-a542-c3279a866b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb0db385bc574121b4fa5748c9099b3e'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.147686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1346/2650] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:26 2022] DELETE /identity/v3/users/bb0db385bc574121b4fa5748c9099b3e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:27.515637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:27.517205 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:27.531760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e0208c-8f45-4f00-a8df-00f01e1ad58c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:27.607351 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:27.608215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.608726 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.609307 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] PATH_INFO: `/v3/projects/7467845b8c774292a85b4ac55eab0993` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.610493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7467845b8c774292a85b4ac55eab0993)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:27.612778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:27.667977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7467845b8c774292a85b4ac55eab0993', 'request_id': 'req-02681173-93c1-4bee-9ef2-f10d922dc77a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.673483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7467845b8c774292a85b4ac55eab0993', 'request_id': 'req-02681173-93c1-4bee-9ef2-f10d922dc77a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.685898 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7467845b8c774292a85b4ac55eab0993', 'request_id': 'req-02681173-93c1-4bee-9ef2-f10d922dc77a'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.697832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] The token cache is being invalidate because project 7467845b8c774292a85b4ac55eab0993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:27.698119 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.698751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-02681173-93c1-4bee-9ef2-f10d922dc77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:27.700903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1305/2651] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:27 2022] DELETE /identity/v3/projects/7467845b8c774292a85b4ac55eab0993 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:27.705862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.706338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.706453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.725182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:27.726380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:27.789812 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1347/2652] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:27.798818 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.799102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.799379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.818976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:27.820165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:27.829378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1306/2653] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:27.833681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:27.843704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee421bb-7d67-425f-945f-504cfac79ccd None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:27.849240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:27.850339 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.850653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.850930 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.851761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:27.852048 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:27.853720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:27.868346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1348/2654] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 15:55:27.870657 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:27.881220 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079a9cf-2fff-4883-a796-2080f5b5f791 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:27.885284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:27.885943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:27.886217 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:27.886488 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:27.887744 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:27.889718 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:27.920327 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1307/2655] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:27 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:27.921993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:27.932131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15a6d7-442b-41df-9e9b-c95b244f608d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:27.964197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:28.008492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:28.009273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.009623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.009963 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.010727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:28.013288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:28.045708 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1349/2656] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:27 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:28.046739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:28.053769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:28.059357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07550b5-6390-4d30-9e82-f44c2e0b0d96 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:28.070353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:28.071039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.071720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.071720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.072400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:28.074486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:28.097334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1308/2657] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:28.097856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:28.110721 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f18d93-772a-4a0d-a0c1-008173df7f7f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:28.122433 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:28.123338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.124664 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.125860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.130528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:28.132165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb95144-c78f-4a45-bd3e-2f59025f76b3 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:28.145085 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1350/2658] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:28 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:55:28.149285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:28.162366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f122f1-fc30-40d1-b27a-3829cce3f7b1 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:28.168571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:28.169522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.169805 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.170082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] PATH_INFO: `/v3/projects/bc1fbb8218ad4da5a2e2ac6d50bf89e0/users/63d864e3821b4c989b474bd313722f33/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.190306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc1fbb8218ad4da5a2e2ac6d50bf89e0, user_id=63d864e3821b4c989b474bd313722f33, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:28.193116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:28.217129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1309/2659] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:28 2022] PUT /identity/v3/projects/bc1fbb8218ad4da5a2e2ac6d50bf89e0/users/63d864e3821b4c989b474bd313722f33/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:28.222488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.222764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.223044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.242109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:28.243727 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:28.312250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1351/2660] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:55:28.319757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:28.321621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:28.408176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3b30-1f6e-49d8-9bc0-ba41ad6de16e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:28.411836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:28.412697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:28.412976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:28.413252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:28.428489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:28.430303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:28.446067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1310/2661] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:29.996440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:29.997902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:30.014910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae0287-e44d-4754-8851-0a11b46e7d4f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:30.022857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:30.024003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:30.024448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:30.024854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:30.032413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:30.055738 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:30.056794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a106a-f610-4d68-8ef0-44bbb31de5ad service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:30.087544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1352/2662] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:31.899233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:31.900757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:31.920600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a15a4-ecf8-4d76-bc9b-31a0f556ae11 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:31.931801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:31.932665 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:31.932952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:31.933232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] PATH_INFO: `/v3/users/ef3c7fa879774e8d89717ac9ce478ef2` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:31.937397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3c7fa879774e8d89717ac9ce478ef2)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:31.939379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:32.038142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3c7fa879774e8d89717ac9ce478ef2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:32.039081 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3c7fa879774e8d89717ac9ce478ef2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:32.041788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3c7fa879774e8d89717ac9ce478ef2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:32.060129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3c7fa879774e8d89717ac9ce478ef2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:32.090190 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3c7fa879774e8d89717ac9ce478ef2'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:32.113303 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1311/2663] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:31 2022] DELETE /identity/v3/users/ef3c7fa879774e8d89717ac9ce478ef2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:32.119299 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:32.119884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:32.120080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:32.153912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:32.155493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:32.275070 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1353/2664] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:32.289107 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:32.291092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:32.383808 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:32.404009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388fefb-d6d4-4266-b4d3-6f5f4a0b33f7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:32.410678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:32.411347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:32.411624 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:32.413509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:32.425946 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:32.427419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ed2871-8d8a-45d8-ac46-473dd49e4a00 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:32.439256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1312/2665] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:33.314064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:33.318044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:33.333347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00c2a2-8861-4ea4-b167-f0a521f0ffcc None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:33.339540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:33.341570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:33.342045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:33.342547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] PATH_INFO: `/v3/projects/74188d6a458443efa85527510fdc1ac1` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:33.343954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] RBAC: Authorizing `identity:delete_project(project_id=74188d6a458443efa85527510fdc1ac1)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:33.346131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:33.397266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:33.425912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74188d6a458443efa85527510fdc1ac1', 'request_id': 'req-54c8bd69-5a26-413c-af99-3b0cb1ff300a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:33.433501 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74188d6a458443efa85527510fdc1ac1', 'request_id': 'req-54c8bd69-5a26-413c-af99-3b0cb1ff300a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:33.444734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74188d6a458443efa85527510fdc1ac1', 'request_id': 'req-54c8bd69-5a26-413c-af99-3b0cb1ff300a'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:33.462872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] The token cache is being invalidate because project 74188d6a458443efa85527510fdc1ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:33.464169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:33.465786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-54c8bd69-5a26-413c-af99-3b0cb1ff300a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:33.468881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1354/2666] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:33 2022] DELETE /identity/v3/projects/74188d6a458443efa85527510fdc1ac1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:33.474296 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:33.477893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:33.479120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:33.502881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:33.505089 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:33.622837 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1313/2667] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:55:33.632998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:33.633551 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:33.633828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:33.675122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:33.676406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:33.710350 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1355/2668] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:33.717072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:33.739721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843fc5e-d04c-45d9-b44e-55dfb4f19ea2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:33.748079 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:33.755687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:33.756142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:33.756560 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:33.757438 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:33.757834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:33.759616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:33.810031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1314/2669] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:55:33.817289 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:33.831820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44babe6-d7ec-4d32-8d53-138ec4cd2145 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:33.836908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:33.840212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:33.840643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:33.841021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:33.841960 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:33.844029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:33.939956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1356/2670] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:33 2022] POST /identity/v3/projects => generated 359 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:33.956627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:33.990285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2961557-cc95-42f4-b444-b067d477974f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.133590 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.136249 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.210100 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.210948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.211232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.211508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.212573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.215530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e6cf-191c-437f-823f-364f95c85bec admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.251611 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1315/2671] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:33 2022] POST /identity/v3/users => generated 280 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:34.261645 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.262316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.262802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.291673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe552-5e57-46b2-bfc2-7a18ec315c8a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.301445 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.302124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.302410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.302689 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.302979 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] MFA Rules not processed for user `63d864e3821b4c989b474bd313722f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:34.304429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.315482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.338202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.341241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.379980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1357/2672] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:55:34.382796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.400224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976a0cd-ca90-4155-bcf5-ef265f34d99d service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.409139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.410097 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.410542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.410979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.412036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.414573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.438628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1358/2673] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:34.440770 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.452354 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0685119-1e01-4c6e-80cd-d779e207427b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.455112 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:34.470209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.472167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.472747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.473275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.474272 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.478111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.505665 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1359/2674] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:34.510771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.516934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1316/2675] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:34.521721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.523153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.534007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e16b7e-d49b-4a61-8576-8765a0c1d27e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.540461 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.541690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.542241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.543953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] PATH_INFO: `/v3/projects/52b9e67658934c48b32ed64a43339126/users/f5ff1aeea6364ea5b8db199a3108860b/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.554914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612876b-5543-4e46-aca7-c7a13484e3d4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.569711 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.570388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.570673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.570980 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.571860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.572881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b9e67658934c48b32ed64a43339126, user_id=f5ff1aeea6364ea5b8db199a3108860b, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.573860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.576544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448032a2-921e-4e1b-98a7-4795d90a0c78 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.614880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1360/2676] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:34 2022] PUT /identity/v3/projects/52b9e67658934c48b32ed64a43339126/users/f5ff1aeea6364ea5b8db199a3108860b/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:34.619088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.619567 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.620570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.639001 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1317/2677] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:34 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:55:34.641940 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.664023 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] MFA Rules not processed for user `f5ff1aeea6364ea5b8db199a3108860b`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:34.665786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.719529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:34.775277 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1361/2678] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:34.784408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f112e8f-f499-4ee5-aa2e-1fdaf37f74e0 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.813902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.816148 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.818080 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:34.818199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.818199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.818199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.818519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.821304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.862780 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1318/2679] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:34 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:55:34.869482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.880716 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3244f2-ae32-443b-b753-822f7cd36a9f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.889098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.889649 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.896083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.896397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.897339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.908699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.924125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1319/2680] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:34.925857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:34.935901 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8efdcd-4ab9-4b77-a5d8-f681419251b3 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.941440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.942672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.943220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.943684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac18b9b-946a-4ffa-be16-2335c473468b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:34.943981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.948888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:34.950476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:34.950764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:34.951044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] PATH_INFO: `/v3/projects/b924dd4792284fad81f0acaef7083f9b/users/989d6807e2404d759f89a6b91c77e3f0/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:34.963028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.965131 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.973008 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b924dd4792284fad81f0acaef7083f9b, user_id=989d6807e2404d759f89a6b91c77e3f0, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:34.979241 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:34.980876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1362/2681] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:34 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:55:35.004997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.015850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1320/2682] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:34 2022] PUT /identity/v3/projects/b924dd4792284fad81f0acaef7083f9b/users/989d6807e2404d759f89a6b91c77e3f0/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:35.017871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.037144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc66c74-b627-4239-bc43-3ccc5ba01ba9 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.143755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f776b9a-f83f-48f8-a885-fd92d4c18614 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.153703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.154501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.154850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.155216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.156334 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.158459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.160707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.161729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.162127 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.165324 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] PATH_INFO: `/v3/users/f5ff1aeea6364ea5b8db199a3108860b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.166711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ff1aeea6364ea5b8db199a3108860b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.169022 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.180993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1321/2683] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:35.180993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.183618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.227968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:35.247748 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ff1aeea6364ea5b8db199a3108860b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.248854 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ff1aeea6364ea5b8db199a3108860b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.249559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ff1aeea6364ea5b8db199a3108860b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.258248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ff1aeea6364ea5b8db199a3108860b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.271217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ff1aeea6364ea5b8db199a3108860b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.290152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:35.292802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e33ec9-cb2c-4f25-afbf-1e510c470674 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.299396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.300491 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1363/2684] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:34 2022] DELETE /identity/v3/users/f5ff1aeea6364ea5b8db199a3108860b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:35.301573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.302781 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.310541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.310541 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.310859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.313680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.326436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.328054 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.342758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1322/2685] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:35.350927 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.353773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.384525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b77a95-32ac-46f1-884e-e3bc81083a67 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.390505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.391689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.393723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.393956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] PATH_INFO: `/v3/users/2d8a7414fd254ec981d35b417dc373cd` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.395754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d8a7414fd254ec981d35b417dc373cd)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.396932 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaac3cf-9fab-485d-ac4e-5096a2eaf578 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.401503 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.402072 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.403093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.403434 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.403777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] PATH_INFO: `/v3/projects/b924dd4792284fad81f0acaef7083f9b/users/989d6807e2404d759f89a6b91c77e3f0/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.405907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b924dd4792284fad81f0acaef7083f9b, user_id=989d6807e2404d759f89a6b91c77e3f0, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.408716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989ca16e-eee2-49d3-93f2-91afc69a048f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.423270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1364/2686] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:35 2022] PUT /identity/v3/projects/b924dd4792284fad81f0acaef7083f9b/users/989d6807e2404d759f89a6b91c77e3f0/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:35.425101 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.425862 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.426223 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.446224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:35.447540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.460417 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d8a7414fd254ec981d35b417dc373cd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.461195 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d8a7414fd254ec981d35b417dc373cd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.462380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d8a7414fd254ec981d35b417dc373cd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.471525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d8a7414fd254ec981d35b417dc373cd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.489206 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d8a7414fd254ec981d35b417dc373cd'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.514752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1323/2687] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:35 2022] DELETE /identity/v3/users/2d8a7414fd254ec981d35b417dc373cd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:35.515912 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.518116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.530494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1365/2688] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:35.531987 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.534039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.551855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af83ca-9e55-4be0-b27b-db65d866498e None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.627583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee5de0-88aa-4365-9695-5081e2aa7941 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.634405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.635193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.635574 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.636155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.638893 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.658400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.660494 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.660690 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.661651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.661937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.662230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] PATH_INFO: `/v3/users/e754dd9099b54fa4ac8debfbc4a645b9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.663771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e754dd9099b54fa4ac8debfbc4a645b9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.665941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.676504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1324/2689] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:35.677499 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:35.679152 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.697354 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:35.724677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:35.753601 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e754dd9099b54fa4ac8debfbc4a645b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.755795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e754dd9099b54fa4ac8debfbc4a645b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.756644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e754dd9099b54fa4ac8debfbc4a645b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.770564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e754dd9099b54fa4ac8debfbc4a645b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.782456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e754dd9099b54fa4ac8debfbc4a645b9'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:35.798317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1d423-ab47-4708-a6ec-9cdb6becfaea service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:35.807618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:35.808487 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:35.808836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:35.809612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:35.813887 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:35.818485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1366/2690] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:55:35 2022] DELETE /identity/v3/users/e754dd9099b54fa4ac8debfbc4a645b9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:35.837209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:35.839154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:35.866760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1325/2691] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:36.056743 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.058718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.192122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2915e7-703c-4302-bbc5-75ac776ec089 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.198216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.199832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.200257 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.200762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] PATH_INFO: `/v3/projects/e8d86717f1954ab7a59cdef0cf13556c` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.202535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d86717f1954ab7a59cdef0cf13556c)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.204904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.305099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d86717f1954ab7a59cdef0cf13556c', 'request_id': 'req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.318781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.319774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d86717f1954ab7a59cdef0cf13556c', 'request_id': 'req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.321977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.334742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d86717f1954ab7a59cdef0cf13556c', 'request_id': 'req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.345046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c51d3-6122-41fb-960b-d7eddbcffd8c service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.352274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] The token cache is being invalidate because project e8d86717f1954ab7a59cdef0cf13556c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:36.352697 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.353393 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.359562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1367/2692] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:36 2022] DELETE /identity/v3/projects/e8d86717f1954ab7a59cdef0cf13556c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:36.360706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.363872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.438072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.441672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.441966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.442555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] PATH_INFO: `/v3/projects/52b9e67658934c48b32ed64a43339126` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.444141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b9e67658934c48b32ed64a43339126)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.447455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.450826 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:36.486227 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae49a3-dd9c-4191-8d6d-4dddf22c26ac admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.494902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.496388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.497158 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.497625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] PATH_INFO: `/v3/projects/44674730e48240da874cc5fb9662455d` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.499192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44674730e48240da874cc5fb9662455d)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.501548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.523291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b9e67658934c48b32ed64a43339126', 'request_id': 'req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.533958 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b9e67658934c48b32ed64a43339126', 'request_id': 'req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.547619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b9e67658934c48b32ed64a43339126', 'request_id': 'req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.580041 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] The token cache is being invalidate because project 52b9e67658934c48b32ed64a43339126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:36.581513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.583127 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d09de85f-4ed0-4ad0-993e-54cc0c8b2e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.588122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1326/2693] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:36 2022] DELETE /identity/v3/projects/52b9e67658934c48b32ed64a43339126 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:36.592259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ce7f-f0d7-496a-a7e4-d68fb9d25d2a None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.594650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ce7f-f0d7-496a-a7e4-d68fb9d25d2a None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.594650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ce7f-f0d7-496a-a7e4-d68fb9d25d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.605014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44674730e48240da874cc5fb9662455d', 'request_id': 'req-7d62a730-793e-45cd-86cb-8fdf546cdd13'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.624715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-5143ce7f-f0d7-496a-a7e4-d68fb9d25d2a None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.626923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5143ce7f-f0d7-496a-a7e4-d68fb9d25d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.635546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44674730e48240da874cc5fb9662455d', 'request_id': 'req-7d62a730-793e-45cd-86cb-8fdf546cdd13'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.650829 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44674730e48240da874cc5fb9662455d', 'request_id': 'req-7d62a730-793e-45cd-86cb-8fdf546cdd13'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.669570 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] The token cache is being invalidate because project 44674730e48240da874cc5fb9662455d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:55:36.670285 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.671366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-7d62a730-793e-45cd-86cb-8fdf546cdd13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:55:36.673591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1368/2694] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:55:36 2022] DELETE /identity/v3/projects/44674730e48240da874cc5fb9662455d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:36.675132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d598c5-2763-471e-992a-ca61f2a81ddb None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.676692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d598c5-2763-471e-992a-ca61f2a81ddb None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.676692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d598c5-2763-471e-992a-ca61f2a81ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.704609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-79d598c5-2763-471e-992a-ca61f2a81ddb None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.706377 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-79d598c5-2763-471e-992a-ca61f2a81ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.720879 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:36.771727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1327/2695] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.773868 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251535-74a9-446e-a3b2-ac8c21395e6c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.774485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251535-74a9-446e-a3b2-ac8c21395e6c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.775056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251535-74a9-446e-a3b2-ac8c21395e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.799084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f8251535-74a9-446e-a3b2-ac8c21395e6c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.803166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f8251535-74a9-446e-a3b2-ac8c21395e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.813215 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1328/2696] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.813990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1369/2697] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.815285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e359-fcf1-481d-9cbc-a3e98a7f3a3d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.815796 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e359-fcf1-481d-9cbc-a3e98a7f3a3d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.816157 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0e359-fcf1-481d-9cbc-a3e98a7f3a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.821848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.828802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.829464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.842045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-f9a0e359-fcf1-481d-9cbc-a3e98a7f3a3d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.843500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f9a0e359-fcf1-481d-9cbc-a3e98a7f3a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.852961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1329/2698] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.854251 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.854614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.854963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.868614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.870733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.879863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:36.881569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:36.885155 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1370/2699] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.894929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.896409 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1330/2700] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:36.897518 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.910092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d70bb5-03a2-406d-b07c-398a8d65f869 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.914555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87516bd-59f0-4bb8-b0e9-8fee20293a24 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.925661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.926965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.927248 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.927550 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.929578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:36.929895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.931553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.936145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.936449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.936682 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.936883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.937631 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:36.937941 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.940531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.958789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1331/2701] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:55:36.960340 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.967765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1371/2702] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:55:36.968729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:36.979903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a81208-5f1b-4bf5-a345-7e4de9b8e331 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.980275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2bb08-8b8e-4c91-97a7-28875c619baa admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:36.988090 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.989998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.990527 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.990970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.991442 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:36.992150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:55:36.992321 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:36.992881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:36.993215 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:36.993401 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.994035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:36.996769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:36.996916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.014147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1332/2703] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:55:37.015304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.030953 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76554d-bfd5-4601-85a1-683f0e0feb26 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.037748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.038882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.039018 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1372/2704] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:36 2022] POST /identity/v3/projects => generated 389 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.039455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.040651 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.040921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.042142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.044812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.109356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1333/2705] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.110116 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.120110 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:37.165993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38fc68-a819-4b66-9c33-9d4dd2a7f743 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.210066 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5fbf5-71d9-48eb-ae24-740536bf331e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.215652 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.216355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.216693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.218701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.219109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.222040 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.245966 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.247528 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.247996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.248543 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.249456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.252422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.252667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1334/2706] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.253859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.269289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59322120-3beb-4ff1-ab9b-4121e34d9f17 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.276333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.276941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.277386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.277604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.278443 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.280935 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.294557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1373/2707] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/projects => generated 377 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.294557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.319723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1335/2708] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/users => generated 287 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.319723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.363286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa0404-eab6-4f02-bb22-a32cc3b9b8f7 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.409248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e8fcc2-c6f3-4f53-9bca-0e4498adf39d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.422196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.423005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.423368 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.423720 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.425415 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.428341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.446798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1374/2709] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.446798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.455573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:37.467465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d5a01-afd4-4ad4-922a-af7fb20e92f9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.474391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.476132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.476530 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.476873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.479667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.482555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.495769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.498683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.500664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.501239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.501989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.505424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.506275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1375/2710] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.507250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.527877 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d332be-9d78-40c6-9d2f-0fe7c59e08f2 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.536644 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.538860 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.539154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.539429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.540657 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.545851 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1336/2711] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:37.546367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.546478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.565524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1376/2712] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.565524 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.568493 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12339324-7f86-4327-a4c0-92c7d936188e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.580428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2872ba-d3bc-4bbb-a260-65d350136dc9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.585655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.586628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.586961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.587360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.587827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.588169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.588379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.588597 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.589447 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.590648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.594621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.595850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.608282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1337/2713] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.609410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.623169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1377/2714] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.623169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.626815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b46856-6b2e-4307-9e85-5949c4bef711 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.632497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.634533 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.634988 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.635120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a924606-5123-400b-b056-d705de1b069e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.635562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] PATH_INFO: `/v3/projects/02862398339b4dbb8cb30bad7abb42b4/users/fb3c8f4ce4a54d338094178f38f689ca/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.643009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.644183 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.645068 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.645443 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] PATH_INFO: `/v3/projects/4f881279f9034a6ba97940ee78b14f29/users/2f056888ef534114be5249b484fc8946/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.659226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=02862398339b4dbb8cb30bad7abb42b4, user_id=fb3c8f4ce4a54d338094178f38f689ca, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.662391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.682236 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1338/2715] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:37 2022] PUT /identity/v3/projects/02862398339b4dbb8cb30bad7abb42b4/users/fb3c8f4ce4a54d338094178f38f689ca/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:37.685782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f881279f9034a6ba97940ee78b14f29, user_id=2f056888ef534114be5249b484fc8946, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.690210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f05f017-a322-40df-8286-72be8f82c328 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.694305 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.724811 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1378/2716] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:37 2022] PUT /identity/v3/projects/4f881279f9034a6ba97940ee78b14f29/users/2f056888ef534114be5249b484fc8946/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:37.726694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809700e9-733f-4b79-a216-699bbc0fbbfe None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.727366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809700e9-733f-4b79-a216-699bbc0fbbfe None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.727752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809700e9-733f-4b79-a216-699bbc0fbbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.736500 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:37.768504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-809700e9-733f-4b79-a216-699bbc0fbbfe None None] MFA Rules not processed for user `fb3c8f4ce4a54d338094178f38f689ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:37.770066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-809700e9-733f-4b79-a216-699bbc0fbbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:37.778379 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab7799-ea7a-49eb-98cf-6f1889430ead admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:37.872961 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1379/2717] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:37.874798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.875384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.875892 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.896694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:37.897766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:37.898052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:37.898324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:37.899062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:37.903321 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:37.924626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1339/2718] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:37.926323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:37.928479 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:37.930094 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] MFA Rules not processed for user `2f056888ef534114be5249b484fc8946`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:37.931831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:38.033955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:38.044369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2f5f-1057-40d9-9fc1-fc179694b494 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.050938 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.052000 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.052513 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.052894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.069391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1380/2719] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:38.069391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.080897 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.084862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.097281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef35c3-bc55-40e6-89be-774d9d346b6d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.113653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.113915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1340/2720] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:38.114593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.115035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.115429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] PATH_INFO: `/v3/projects/64c7288b683a4d888132ae19f9f0187b/users/f2aa158c46cf4e4895ff578db093e80c/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.116245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.138579 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c7288b683a4d888132ae19f9f0187b, user_id=f2aa158c46cf4e4895ff578db093e80c, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.141853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cd6d5-3f9c-412c-8e28-ef6417e98199 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.149257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad3ae7-3523-4c97-8412-f9a748b2a2dc service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.155245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.156009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.156393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.156681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.157410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.159775 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.173130 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1381/2721] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:38 2022] PUT /identity/v3/projects/64c7288b683a4d888132ae19f9f0187b/users/f2aa158c46cf4e4895ff578db093e80c/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:38.175759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.176082 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.176346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.208575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1341/2722] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:55:38 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:38.212530 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.227309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] MFA Rules not processed for user `f2aa158c46cf4e4895ff578db093e80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:38.229594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:38.343973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:38.349019 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129bc6e-37a8-456c-bb97-75ddfaf8e624 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.367806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.368562 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.368863 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.370577 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.374573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.380655 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.402330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1382/2723] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:38.415466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.417953 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:38.425804 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1342/2724] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:55:38 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:55:38.427846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.468347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b8022-017d-4be8-956e-270637d2fbac admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.474358 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.475174 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.475664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.475943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.478543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.481287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.509825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1343/2725] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:38.514512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.549134 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecc80d8-4cbd-4f34-b1df-9a5439790428 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.578157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff21567-b1f6-4a99-bec1-fe187221cbd8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.585005 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.587834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.588069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.594199 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] PATH_INFO: `/v3/projects/649a758795084b5fa10f371c7d185e7d/users/a78e4ca8184049089ae004557f008eb4/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.612193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.620555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.622209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.623612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.651966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=649a758795084b5fa10f371c7d185e7d, user_id=a78e4ca8184049089ae004557f008eb4, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.658257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.687887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.687887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.715122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1344/2726] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:38 2022] PUT /identity/v3/projects/649a758795084b5fa10f371c7d185e7d/users/a78e4ca8184049089ae004557f008eb4/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:38.719084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.723958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1383/2727] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:38.858960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35f447-d629-463f-84c2-c37409914fd8 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.865014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.865844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.866219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.866703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.869281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.872540 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee5cfe-08ec-4480-a800-ea52c2501ee0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.897950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1345/2728] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:55:38 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:55:38.899294 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:38.915587 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6266ad0-3e0b-4d5c-8ead-64cb1cc6f71b service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:38.923409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:38.925125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.925125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.925125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] PATH_INFO: `/v3/projects/649a758795084b5fa10f371c7d185e7d/users/a78e4ca8184049089ae004557f008eb4/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.927100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=649a758795084b5fa10f371c7d185e7d, user_id=a78e4ca8184049089ae004557f008eb4, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:38.930449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:38.945610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1384/2729] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:55:38 2022] PUT /identity/v3/projects/649a758795084b5fa10f371c7d185e7d/users/a78e4ca8184049089ae004557f008eb4/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:55:38.955776 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:38.956698 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:38.957168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:38.995382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] MFA Rules not processed for user `a78e4ca8184049089ae004557f008eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:38.997501 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:39.004374 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:39.099870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1346/2730] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:39.112666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:39.234390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e065a7bd-911c-4473-88d9-02f818171abe admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:39.250086 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:39.251009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:39.251262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:39.251529 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:39.280348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:39.281822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20096f5b-ab6b-4f14-9933-26eddbb3c2f3 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:39.319667 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1385/2731] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:39 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:39.433050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:39.453425 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f7e90-9b64-49e3-bf9d-9de6ba009390 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:39.459792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:39.460763 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:39.461052 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:39.471465 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:39.617259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:39.625114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:39.655097 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1347/2732] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:41.778956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e7ab-0996-4262-84c9-2c4d0c58370e None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:41.779458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e7ab-0996-4262-84c9-2c4d0c58370e None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:41.779679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635e7ab-0996-4262-84c9-2c4d0c58370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:41.828800 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-6635e7ab-0996-4262-84c9-2c4d0c58370e None None] MFA Rules not processed for user `989d6807e2404d759f89a6b91c77e3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:41.830106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-6635e7ab-0996-4262-84c9-2c4d0c58370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:41.919163 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1386/2733] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:41.935989 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:55:41.941113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:42.038256 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd1f67-e20c-4ad9-be16-ebd43b4727a4 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:55:42.046803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:55:42.047752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:42.049028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:42.050257 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:42.053474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:42.115196 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:42.138181 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:55:42.139820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:55:42.149528 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1348/2734] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:55:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:55:43.077230 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393e2da-d169-475d-8f4a-4506208806ca None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:55:43.077719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393e2da-d169-475d-8f4a-4506208806ca None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:55:43.077827 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a393e2da-d169-475d-8f4a-4506208806ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:55:43.089100 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:43.098511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a393e2da-d169-475d-8f4a-4506208806ca None None] MFA Rules not processed for user `63d864e3821b4c989b474bd313722f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:55:43.099944 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a393e2da-d169-475d-8f4a-4506208806ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:55:43.107273 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1387/2735] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:55:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:55:52.145596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=10 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:55:53.095632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:07.380689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:07.442186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef6315-e79b-460f-a867-00e510f999fb service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:07.446601 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:07.447462 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:07.447738 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:07.448004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] PATH_INFO: `/v3/users/fb3c8f4ce4a54d338094178f38f689ca` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:07.449189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3c8f4ce4a54d338094178f38f689ca)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:07.451105 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:07.496529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3c8f4ce4a54d338094178f38f689ca'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:07.497179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3c8f4ce4a54d338094178f38f689ca'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:07.497789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3c8f4ce4a54d338094178f38f689ca'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:07.505151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3c8f4ce4a54d338094178f38f689ca'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:07.517397 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3c8f4ce4a54d338094178f38f689ca'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:07.529458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1349/2736] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:56:07 2022] DELETE /identity/v3/users/fb3c8f4ce4a54d338094178f38f689ca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:07.533844 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:07.534305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:07.534416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:07.551305 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:07.552522 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:07.609683 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1388/2737] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:56:07.619529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:07.621092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:07.679544 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fadf5e-2120-4069-b6f6-8d109f882ce5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:07.683267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:07.683942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:07.684218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:07.684520 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:07.694154 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:07.695836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:07.705948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1350/2738] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:07.973670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:07.975200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:07.989552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6988e-ade8-4d39-988a-f93984076bfc None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:07.994518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:07.995386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:07.995694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:07.995977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] PATH_INFO: `/v3/projects/02862398339b4dbb8cb30bad7abb42b4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:07.997345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02862398339b4dbb8cb30bad7abb42b4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:07.999300 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:08.048375 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02862398339b4dbb8cb30bad7abb42b4', 'request_id': 'req-4df35968-fb67-49a9-b6af-05e6908e0c81'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:08.053943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02862398339b4dbb8cb30bad7abb42b4', 'request_id': 'req-4df35968-fb67-49a9-b6af-05e6908e0c81'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:08.061268 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02862398339b4dbb8cb30bad7abb42b4', 'request_id': 'req-4df35968-fb67-49a9-b6af-05e6908e0c81'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:08.072527 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] The token cache is being invalidate because project 02862398339b4dbb8cb30bad7abb42b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:56:08.072816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:08.073449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:08.074996 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1389/2739] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:56:07 2022] DELETE /identity/v3/projects/02862398339b4dbb8cb30bad7abb42b4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:12.216884 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:12.226098 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe691f52-9164-4210-9758-9e2287337dc3 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:12.244382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:12.277131 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:12.277766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:12.278044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:12.278319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:12.280077 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:12.289764 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:12.291227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:12.299136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1351/2740] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:12.762020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:12.763677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:12.859653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df35968-fb67-49a9-b6af-05e6908e0c81 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:12.865935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2000c52a-6412-4503-9642-462025c26697 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:12.867014 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c52a-6412-4503-9642-462025c26697 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:12.867462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c52a-6412-4503-9642-462025c26697 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:12.868318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2000c52a-6412-4503-9642-462025c26697 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:12.871572 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2000c52a-6412-4503-9642-462025c26697 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:12.885766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000c52a-6412-4503-9642-462025c26697 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:12.887370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2000c52a-6412-4503-9642-462025c26697 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:12.899531 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1390/2741] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:15.813753 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:15.815189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:15.884589 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab885b50-26ab-4ac0-9c45-ae85c0f056f8 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:15.889084 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:15.889956 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:15.890254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:15.890532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] PATH_INFO: `/v3/users/f2aa158c46cf4e4895ff578db093e80c` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:15.891724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2aa158c46cf4e4895ff578db093e80c)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:15.893684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:15.942408 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2aa158c46cf4e4895ff578db093e80c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:15.943064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2aa158c46cf4e4895ff578db093e80c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:15.943664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2aa158c46cf4e4895ff578db093e80c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:15.950613 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2aa158c46cf4e4895ff578db093e80c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:15.962792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2aa158c46cf4e4895ff578db093e80c'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:15.976869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1352/2742] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:56:15 2022] DELETE /identity/v3/users/f2aa158c46cf4e4895ff578db093e80c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:15.981420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:15.981916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:15.981993 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.000515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:16.001919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:16.010208 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:16.067248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1391/2743] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:16.076870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:16.078823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:16.128727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:16.162076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d24c7-20a6-475f-8b5a-acffc9f0e735 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:16.165797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:16.166439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.166721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.166995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.176529 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:16.177985 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30ff617-35a1-4d07-92e0-bd12bb5fab43 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:16.188017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1353/2744] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:16.560666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:16.562666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:16.588654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a8570-b542-42d7-b2a7-213cd2ec7caa None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:16.593752 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:16.595118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.595408 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.595685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] PATH_INFO: `/v3/projects/64c7288b683a4d888132ae19f9f0187b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.596897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c7288b683a4d888132ae19f9f0187b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:16.598851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:16.647280 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c7288b683a4d888132ae19f9f0187b', 'request_id': 'req-a7302820-558f-4967-92ee-83f496960189'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:16.652796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c7288b683a4d888132ae19f9f0187b', 'request_id': 'req-a7302820-558f-4967-92ee-83f496960189'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:16.660773 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c7288b683a4d888132ae19f9f0187b', 'request_id': 'req-a7302820-558f-4967-92ee-83f496960189'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:16.672261 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] The token cache is being invalidate because project 64c7288b683a4d888132ae19f9f0187b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:56:16.672615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:16.673241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a7302820-558f-4967-92ee-83f496960189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:16.674799 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1392/2745] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:56:16 2022] DELETE /identity/v3/projects/64c7288b683a4d888132ae19f9f0187b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:16.681219 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.681535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.681814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.700564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:16.701789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:16.768810 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1354/2746] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:16.776714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.776997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.777282 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.797184 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:16.798065 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:16.806609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1393/2747] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:16.812559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:16.822982 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a7ff7-daa6-41c6-8aeb-6b7aa9d567b9 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:16.828504 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:16.829202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.829482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.829754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.830486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:56:16.830793 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:16.832494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:16.846889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1355/2748] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:56:16.851548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:16.861921 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24953e8c-21dd-48ce-a4f6-8c75353afaa0 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:16.867236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:16.868474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.868751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.869171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.870021 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:16.872106 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:16.910186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1394/2749] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:56:16 2022] POST /identity/v3/projects => generated 397 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:16.913998 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:16.925317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb223be-015c-480b-8fda-5e6e29488f9a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:16.986815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:16.987480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:16.987760 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:16.988032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:16.988835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:16.990815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:17.013286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1356/2750] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:56:16 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:17.017367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:17.027837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f53b1-1431-4649-b075-745ae9374c90 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:17.034897 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:17.036006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:17.036588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:17.037024 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:17.038291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:17.040395 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:17.051983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1395/2751] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:17.055899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:17.066664 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116abe70-f700-42e5-84bf-88489be08829 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:17.072232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:17.072947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:17.073232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:17.073510 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:17.074245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:17.075869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32bfdb-8ba7-4161-9448-4e29ae4cf49a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:17.086788 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1357/2752] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:17.090704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:17.101241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420a815-4886-4f3f-bb63-22f63be2329b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:17.105885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:17.106768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:17.107050 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:17.107325 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] PATH_INFO: `/v3/projects/145dd4f41800444cbbd0c9939ab5c1c9/users/c7cc6e48d4f044e5abb51c0f30a6460e/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:17.132384 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] RBAC: Authorizing `identity:create_grant(project_id=145dd4f41800444cbbd0c9939ab5c1c9, user_id=c7cc6e48d4f044e5abb51c0f30a6460e, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:17.135173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:17.149388 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1396/2753] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:17 2022] PUT /identity/v3/projects/145dd4f41800444cbbd0c9939ab5c1c9/users/c7cc6e48d4f044e5abb51c0f30a6460e/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:17.160870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:17.161101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:17.161383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:17.188758 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] MFA Rules not processed for user `c7cc6e48d4f044e5abb51c0f30a6460e`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:17.189981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:17.211923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:17.257627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1358/2754] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:17.268906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:17.270615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:17.305467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:17.327866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8fde-ae71-432a-93fd-7538d821370f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:17.331466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:17.332104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:17.332412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:17.332684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:17.340712 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:17.342118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ae802-9daf-4d55-aaa2-d1e70784252b service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:17.351139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1397/2755] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:27.254686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=17 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:27.347291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=11 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:29.572845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:29.634104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24492992-7a8c-4dbe-a078-ddc4a2694932 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:29.638632 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:29.639491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:29.639774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:29.640051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] PATH_INFO: `/v3/users/c7cc6e48d4f044e5abb51c0f30a6460e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:29.641253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7cc6e48d4f044e5abb51c0f30a6460e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:29.643756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:29.688086 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7cc6e48d4f044e5abb51c0f30a6460e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:29.688741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7cc6e48d4f044e5abb51c0f30a6460e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:29.689338 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7cc6e48d4f044e5abb51c0f30a6460e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:29.696125 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7cc6e48d4f044e5abb51c0f30a6460e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:29.709060 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7cc6e48d4f044e5abb51c0f30a6460e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:29.726117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1359/2756] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:56:29 2022] DELETE /identity/v3/users/c7cc6e48d4f044e5abb51c0f30a6460e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:29.730290 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:29.730709 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:29.730840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:29.753406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:29.754585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:29.814074 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1398/2757] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:29.824053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:29.825517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:29.889870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2f95-6c6a-4104-871f-f9bb17816f09 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:29.894534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:29.895182 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:29.895459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:29.895729 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:29.904795 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:29.906235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9d8fd-71ad-4a1e-b1d5-e49affd1da14 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:29.915866 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1360/2758] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:30.183400 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.184876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:30.196744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea2acd4-3649-42e1-9ef7-570e0dcc30fe None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.202030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.203059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.203344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.203686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] PATH_INFO: `/v3/projects/145dd4f41800444cbbd0c9939ab5c1c9` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.205046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145dd4f41800444cbbd0c9939ab5c1c9)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.208626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.267378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145dd4f41800444cbbd0c9939ab5c1c9', 'request_id': 'req-e23de2f9-46f2-4f51-9b26-722907b52d61'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:30.274778 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145dd4f41800444cbbd0c9939ab5c1c9', 'request_id': 'req-e23de2f9-46f2-4f51-9b26-722907b52d61'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:30.283203 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145dd4f41800444cbbd0c9939ab5c1c9', 'request_id': 'req-e23de2f9-46f2-4f51-9b26-722907b52d61'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:30.295199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] The token cache is being invalidate because project 145dd4f41800444cbbd0c9939ab5c1c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:56:30.295485 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:30.296129 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e23de2f9-46f2-4f51-9b26-722907b52d61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:30.297716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1399/2759] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:56:30 2022] DELETE /identity/v3/projects/145dd4f41800444cbbd0c9939ab5c1c9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:30.303293 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.303630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.303916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.323013 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:30.324292 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:30.344740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:30.395045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1361/2760] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:30.403036 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.403390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.403733 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.423767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:30.425595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:30.434704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1400/2761] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:30.438959 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.449163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058d0c7-869d-4be6-a2cc-31fd4b3f765d None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.453633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.454274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.454552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.454824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.455539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:56:30.455847 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.457483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.468872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1362/2762] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:56:30.473034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.482263 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332bc68-5217-4d3d-a008-42a484c72147 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.487466 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.488143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.488421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.488700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.489398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.491292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.516507 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1401/2763] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:56:30 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:30.521894 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.531950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfdd3a-6d45-4c62-b781-1e48803b50d2 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.588536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.589193 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.589494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.589765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.590441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.592380 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.612497 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1363/2764] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:56:30 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:56:30.616581 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.627120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959309a-a218-48e2-9f08-6cc61f94aa44 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.631679 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.632355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.632638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.632909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.633629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.635206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.645329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1402/2765] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:30.649017 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.659267 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96724938-d272-47f3-b760-ca9cb87bc12c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.663855 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.664542 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.664825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.665103 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.665823 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.667419 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916cffe6-69a4-4a37-af6d-09832fc56abe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.677756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1364/2766] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:30.681439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.690973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6035e6-8de1-47c6-8c51-9341c0e69c0a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.695413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.696266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.696577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.696847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] PATH_INFO: `/v3/projects/f5b185a216134b4d828add9ebdcf89d2/users/44b73e975868496fb800c68e2d007faa/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.710684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b185a216134b4d828add9ebdcf89d2, user_id=44b73e975868496fb800c68e2d007faa, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.713336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.726882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1403/2767] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:30 2022] PUT /identity/v3/projects/f5b185a216134b4d828add9ebdcf89d2/users/44b73e975868496fb800c68e2d007faa/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:30.731687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.732045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.732363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.750841 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:30.752028 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:30.809628 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1365/2768] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:30.819307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.821291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:30.844323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:30.887865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caef50ff-7f97-442a-a1f7-b194c44870da admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:30.891589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:30.892244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:30.892565 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:30.893347 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:30.903345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:30.904858 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:30.916067 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1404/2769] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:30.989233 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:30.991062 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:31.045895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:31.178429 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d6027-f7d3-45ad-9d05-8c69ddaa0d59 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:31.186021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:31.186767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:31.187123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:31.187453 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:31.189569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:31.204483 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:31.206827 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:31.219705 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1366/2770] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:34.546734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:34.548795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:34.563797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17702499-2f9d-484d-bd32-2773f46b2f10 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:34.570481 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:34.571383 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:34.572563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:34.572912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:34.574885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:34.623509 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:34.643568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28c47c-8066-4ce7-bf6c-467cd134d1ee service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:34.647701 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:34.648378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:34.648661 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:34.648934 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:34.651948 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:34.672025 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:34.673833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa35d06f-7fb2-4820-8d94-aaf7fb65fcfd service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:34.684669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:34.686508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1367/2771] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:56:34.689153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:34.709584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1405/2772] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:36.302216 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.302703 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.302869 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.333568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] MFA Rules not processed for user `44b73e975868496fb800c68e2d007faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:36.334820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:36.410500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1368/2773] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:36.414788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.425937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5703d-0cf1-4f83-8b1b-1b5aeca1e321 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.430757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.431420 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.431702 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.431977 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.432701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.434611 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.463132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1406/2774] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:56:36 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:36.467138 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.478990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c3b84-1d87-482a-b8cd-6d6c671c831f None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.541285 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.542059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.542344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.542614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.543313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.545301 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.555914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:36.573832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1369/2775] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:56:36 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:36.575286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.591111 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1a61b3-2e62-401c-81bb-4610c3770051 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.596908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.597694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.598107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.598387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.599128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.600925 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.613309 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1407/2776] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:36.617424 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.628123 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e1c05-bbe7-4713-8bd0-915d84f3b992 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.632942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.633899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.634269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.634618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] PATH_INFO: `/v3/projects/69cef26705864bda8ee1cd908c39542b/users/aff96157516f4c07b6d24d8081395ddf/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.652186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69cef26705864bda8ee1cd908c39542b, user_id=aff96157516f4c07b6d24d8081395ddf, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.656151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.677659 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1370/2777] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:36 2022] PUT /identity/v3/projects/69cef26705864bda8ee1cd908c39542b/users/aff96157516f4c07b6d24d8081395ddf/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:36.681492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.691817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f53bf2-3538-4e4e-b4dc-80d5d84e413a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.778933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.779609 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.779908 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.780181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.780981 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.782696 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9edd1-d7ec-484d-9e28-52881f6061cf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.796338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1408/2778] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:56:36.797686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:36.808694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5796b0-dbba-42e8-876e-4a6caca2b1a5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:36.813393 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:36.814274 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:36.814900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:36.814900 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] PATH_INFO: `/v3/projects/69cef26705864bda8ee1cd908c39542b/users/aff96157516f4c07b6d24d8081395ddf/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:36.818024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69cef26705864bda8ee1cd908c39542b, user_id=aff96157516f4c07b6d24d8081395ddf, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:36.821218 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:36.833943 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1371/2779] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:36 2022] PUT /identity/v3/projects/69cef26705864bda8ee1cd908c39542b/users/aff96157516f4c07b6d24d8081395ddf/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:41.969734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562235-345b-4cea-b351-3a5944df465f None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:41.970620 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562235-345b-4cea-b351-3a5944df465f None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:41.971099 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c562235-345b-4cea-b351-3a5944df465f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:42.013769 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-2c562235-345b-4cea-b351-3a5944df465f None None] MFA Rules not processed for user `aff96157516f4c07b6d24d8081395ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:42.015009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2c562235-345b-4cea-b351-3a5944df465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:42.025390 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:42.118103 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1409/2780] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:42.134833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:42.221875 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2156767-7899-4f95-9489-91e6a5056140 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:42.228170 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:42.228867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:42.229147 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:42.229433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:42.312348 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:42.315101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20070fdd-7c65-41c1-b324-bcb0cb96698e service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:42.326139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1372/2781] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:43.397123 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c562235-345b-4cea-b351-3a5944df465f None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:43.404836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2c562235-345b-4cea-b351-3a5944df465f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:43.486439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c562235-345b-4cea-b351-3a5944df465f None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:43.491884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:43.493323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:43.493660 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:43.493916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] PATH_INFO: `/v3/users/2f056888ef534114be5249b484fc8946` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:43.495186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f056888ef534114be5249b484fc8946)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:43.497161 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:43.565357 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f056888ef534114be5249b484fc8946'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:43.566379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f056888ef534114be5249b484fc8946'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:43.567241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f056888ef534114be5249b484fc8946'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:43.581298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f056888ef534114be5249b484fc8946'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:43.603270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f056888ef534114be5249b484fc8946'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:43.624666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1410/2782] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:56:43 2022] DELETE /identity/v3/users/2f056888ef534114be5249b484fc8946 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:43.630302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:43.630807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:43.636360 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:43.653920 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:43.655297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:43.712159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:43.728484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1373/2783] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:43.747578 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:43.749118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:43.784064 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:43.826927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724df6a-5dcb-4293-ae49-5eb4e8b3f066 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:43.832037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:43.832883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:43.833171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:43.833453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:43.844725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:43.847102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:43.861547 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1411/2784] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:44.219484 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:44.221857 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:44.246085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfe97f0-4568-43bc-ab34-7ef65c589587 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:44.256357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:44.257101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:44.257832 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:44.258271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] PATH_INFO: `/v3/projects/4f881279f9034a6ba97940ee78b14f29` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:44.260359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f881279f9034a6ba97940ee78b14f29)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:44.266941 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:44.334820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f881279f9034a6ba97940ee78b14f29', 'request_id': 'req-6484ce92-8887-4819-8ecc-33fe68a11139'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.344269 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f881279f9034a6ba97940ee78b14f29', 'request_id': 'req-6484ce92-8887-4819-8ecc-33fe68a11139'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.357717 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f881279f9034a6ba97940ee78b14f29', 'request_id': 'req-6484ce92-8887-4819-8ecc-33fe68a11139'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.373961 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] The token cache is being invalidate because project 4f881279f9034a6ba97940ee78b14f29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:56:44.374252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.375786 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.380815 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1374/2785] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:56:44 2022] DELETE /identity/v3/projects/4f881279f9034a6ba97940ee78b14f29 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:44.382317 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:44.395838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf98512-e091-4c0a-8e12-2e0768c1564b service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:44.507212 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:44.509232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:44.509678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:44.510281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] PATH_INFO: `/v3/users/a78e4ca8184049089ae004557f008eb4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:44.511734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78e4ca8184049089ae004557f008eb4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:44.513913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:44.577323 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78e4ca8184049089ae004557f008eb4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.578140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78e4ca8184049089ae004557f008eb4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.578900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78e4ca8184049089ae004557f008eb4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.589835 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78e4ca8184049089ae004557f008eb4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.600791 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f6be23b2-b867-4136-94e3-a98dc478b163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78e4ca8184049089ae004557f008eb4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:44.625337 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1412/2786] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:56:44 2022] DELETE /identity/v3/users/a78e4ca8184049089ae004557f008eb4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:56:44.981582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:44.982937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:45.003067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484ce92-8887-4819-8ecc-33fe68a11139 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.100886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.101806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.102085 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.102366 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] PATH_INFO: `/v3/projects/649a758795084b5fa10f371c7d185e7d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.103592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=649a758795084b5fa10f371c7d185e7d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.105745 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.174937 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '649a758795084b5fa10f371c7d185e7d', 'request_id': 'req-19decae9-3814-414f-8cc0-f27d1cd40cf5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:45.190265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '649a758795084b5fa10f371c7d185e7d', 'request_id': 'req-19decae9-3814-414f-8cc0-f27d1cd40cf5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:45.201986 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '649a758795084b5fa10f371c7d185e7d', 'request_id': 'req-19decae9-3814-414f-8cc0-f27d1cd40cf5'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:45.216135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] The token cache is being invalidate because project 649a758795084b5fa10f371c7d185e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:56:45.216519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:45.217253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-19decae9-3814-414f-8cc0-f27d1cd40cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:56:45.220254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1375/2787] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:56:44 2022] DELETE /identity/v3/projects/649a758795084b5fa10f371c7d185e7d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:45.226293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.226467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.226747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.247156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:45.248410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:45.294545 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:45.315855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1413/2788] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:56:45.323676 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.323966 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.324247 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.337265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:45.344032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:45.345742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:45.355539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1376/2789] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:45.359837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.370758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1af80-5a94-4a98-820b-942e87ae0514 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.375603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.376278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.376591 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.377515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.378253 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:56:45.378563 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.380200 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.396464 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1414/2790] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:56:45.400723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.412076 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfab3b1-a6f3-49fd-815e-a2cbdbdc82d2 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.418396 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.419056 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.419339 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.419612 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.420485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.422473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.457835 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1377/2791] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:45.459166 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.469671 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e7856-d71c-4156-b99f-502733a93718 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.542915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.543757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.544046 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.544353 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.545063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.547010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.581254 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1415/2792] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:45.583359 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.598277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeecb82-add6-4553-8bef-a7dc8197477e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.604352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.605038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.605316 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.605588 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.606469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.608093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.624490 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1378/2793] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:56:45.628371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.640490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59ffbe-6f09-4038-9766-27920a458932 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.647356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.647863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.648148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.648607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.649345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.651302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac0a93-cfad-450d-8497-d72cbb7473fe admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.667217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1416/2794] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:45.668734 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.679616 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf029b47-087a-4018-9e27-2fdaf325e546 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.688026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.689969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.690485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.691526 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] PATH_INFO: `/v3/projects/76de23a4f2464691a850dc25612fc087/users/8566a230c5c640018a6d0add6b369629/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.712458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76de23a4f2464691a850dc25612fc087, user_id=8566a230c5c640018a6d0add6b369629, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.715674 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.731928 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1379/2795] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:45 2022] PUT /identity/v3/projects/76de23a4f2464691a850dc25612fc087/users/8566a230c5c640018a6d0add6b369629/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:45.743757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.744164 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.744495 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.778621 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] MFA Rules not processed for user `8566a230c5c640018a6d0add6b369629`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:45.779836 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:45.849341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1417/2796] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:45.853784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.920594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06877dba-1aa9-4a47-a399-6fe3e25ad624 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:45.925367 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:45.926026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:45.926304 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:45.926582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:45.927287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:45.929234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:45.960168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1380/2797] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:45.961865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:45.977898 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbbec4a-7c49-4fd4-a92e-c2ae3285a2d7 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.031634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:46.053801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.054532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.054822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.055102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.055808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.057850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.086822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1418/2798] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:56:45 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:56:46.088032 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:46.103454 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346e8e9-f32c-4d49-a114-41547d04c3ba admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.108706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.109545 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.109880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.110156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.110888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.112623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.123485 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1381/2799] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:56:46.127747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:46.138731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb821bb-40cc-4f92-9246-400b161c8ef9 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.144191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.145153 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.145440 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.145726 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] PATH_INFO: `/v3/projects/e53c527056be4424bc0cfb83c009171d/users/fc1347fb1d584e70b771efede34fe9bd/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.161471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53c527056be4424bc0cfb83c009171d, user_id=fc1347fb1d584e70b771efede34fe9bd, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.164108 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.176766 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1419/2800] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:46 2022] PUT /identity/v3/projects/e53c527056be4424bc0cfb83c009171d/users/fc1347fb1d584e70b771efede34fe9bd/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:46.180970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:46.191563 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca30f8e-3af0-438c-8dd9-9281285c7f7f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.250990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.252192 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.252521 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.252803 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.253568 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.255286 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6264fa-0e8c-4d94-9575-eec9e5f38279 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.269234 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1382/2801] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:56:46.273386 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:46.284410 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32153594-79b3-415e-b169-a0c53cd88895 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.289032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.289910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.290194 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.290473 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] PATH_INFO: `/v3/projects/e53c527056be4424bc0cfb83c009171d/users/fc1347fb1d584e70b771efede34fe9bd/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.292469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53c527056be4424bc0cfb83c009171d, user_id=fc1347fb1d584e70b771efede34fe9bd, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.295112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.305771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1420/2802] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:56:46 2022] PUT /identity/v3/projects/e53c527056be4424bc0cfb83c009171d/users/fc1347fb1d584e70b771efede34fe9bd/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:56:46.365722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.366012 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.366582 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.393505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:56:46.396346 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] MFA Rules not processed for user `fc1347fb1d584e70b771efede34fe9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:56:46.398404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:46.471681 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1383/2803] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:56:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:56:46.483083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:46.484821 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:56:46.543675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227296ff-536c-4b0f-aea8-a3bab35f7995 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:46.547480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:46.548136 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:46.548475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:46.548782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:46.607438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:46.608916 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a4626-8064-4089-9b0a-d833472ea98a service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:46.617997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1421/2804] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:54.306003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:56:54.318580 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d720d-a012-4a36-9a01-a4a27856dba8 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:56:54.322168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:56:54.322756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:56:54.323031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:56:54.323302 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:56:54.332754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:56:54.334189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:56:54.343378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1384/2805] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:56:54 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:56:56.613706 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:04.339308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=34 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:09.386140 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:09.386627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:09.386713 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:09.403922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] MFA Rules not processed for user `44b73e975868496fb800c68e2d007faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:09.405152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:09.459141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1422/2806] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:09.767880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:09.770072 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:09.827975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b650-3489-46e1-94fd-87518008a1c2 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:09.831769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:09.832965 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:09.833242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:09.833519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:09.835385 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:09.845782 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:09.847239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a01df-2a8d-459d-8067-af011578d899 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:09.855508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1385/2807] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:19.451974 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=25 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:19.851811 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=26 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:23.418062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:23.479906 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2030c-ab18-4e76-afd2-0edf4535cad8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:23.484468 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:23.485331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:23.485614 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:23.485888 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] PATH_INFO: `/v3/users/8566a230c5c640018a6d0add6b369629` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:23.487071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8566a230c5c640018a6d0add6b369629)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:23.489558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:23.536342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8566a230c5c640018a6d0add6b369629'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:23.536999 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8566a230c5c640018a6d0add6b369629'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:23.537632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8566a230c5c640018a6d0add6b369629'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:23.544370 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8566a230c5c640018a6d0add6b369629'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:23.551848 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8566a230c5c640018a6d0add6b369629'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:23.569472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1423/2808] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:23 2022] DELETE /identity/v3/users/8566a230c5c640018a6d0add6b369629 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:23.573942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:23.574386 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:23.574473 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:23.591010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:23.592176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:23.649331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1386/2809] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:57:23.658718 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:23.660093 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:23.716700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f24d4b5-7a68-451c-96d7-b4e4d2fe7137 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:23.720239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:23.720900 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:23.721176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:23.721448 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:23.730705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:23.732118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:23.741937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1424/2810] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:57:24.000619 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.002395 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:24.016445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a691a584-85cd-4873-acea-2f3cd71d1cbf None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.021428 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.022331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.022685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.023047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] PATH_INFO: `/v3/projects/76de23a4f2464691a850dc25612fc087` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.024494 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76de23a4f2464691a850dc25612fc087)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.026662 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.081688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76de23a4f2464691a850dc25612fc087', 'request_id': 'req-35ab102b-a80c-4fe3-857c-9986c0cbff56'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.088496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76de23a4f2464691a850dc25612fc087', 'request_id': 'req-35ab102b-a80c-4fe3-857c-9986c0cbff56'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.096313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76de23a4f2464691a850dc25612fc087', 'request_id': 'req-35ab102b-a80c-4fe3-857c-9986c0cbff56'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.111534 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] The token cache is being invalidate because project 76de23a4f2464691a850dc25612fc087 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:24.111825 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.112515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.114113 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1387/2811] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:23 2022] DELETE /identity/v3/projects/76de23a4f2464691a850dc25612fc087 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:24.118439 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.128453 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835219a-b9fb-4389-91cd-3d1b85400ab5 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.147478 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:24.191306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.192191 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.192488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.192761 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] PATH_INFO: `/v3/users/fc1347fb1d584e70b771efede34fe9bd` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.193946 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1347fb1d584e70b771efede34fe9bd)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.195887 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.239119 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1347fb1d584e70b771efede34fe9bd'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.239745 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1347fb1d584e70b771efede34fe9bd'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.240361 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1347fb1d584e70b771efede34fe9bd'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.247217 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1347fb1d584e70b771efede34fe9bd'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.254035 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-59028281-262d-4dc3-910d-004c817e682b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1347fb1d584e70b771efede34fe9bd'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.271199 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1425/2812] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:24 2022] DELETE /identity/v3/users/fc1347fb1d584e70b771efede34fe9bd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:24.538179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.539614 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:24.553610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ab102b-a80c-4fe3-857c-9986c0cbff56 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.610421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.611275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.611554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.611831 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] PATH_INFO: `/v3/projects/e53c527056be4424bc0cfb83c009171d` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.613009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53c527056be4424bc0cfb83c009171d)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.615539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.644535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:24.662111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53c527056be4424bc0cfb83c009171d', 'request_id': 'req-1494467c-1689-4b3b-85c7-f791e746f057'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.667486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53c527056be4424bc0cfb83c009171d', 'request_id': 'req-1494467c-1689-4b3b-85c7-f791e746f057'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.674363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53c527056be4424bc0cfb83c009171d', 'request_id': 'req-1494467c-1689-4b3b-85c7-f791e746f057'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.686437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] The token cache is being invalidate because project e53c527056be4424bc0cfb83c009171d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:24.686721 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.687814 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-1494467c-1689-4b3b-85c7-f791e746f057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:24.689371 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1388/2813] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:24 2022] DELETE /identity/v3/projects/e53c527056be4424bc0cfb83c009171d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:24.694373 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.694817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.694905 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.711808 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:24.713071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:24.769681 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1426/2814] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:24.777014 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.777472 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.777812 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.797391 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:24.798561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:24.806955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1389/2815] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:24.811105 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.820904 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10d38c-395f-4d59-91a1-8780f649f424 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.825482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.826135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.826419 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.826698 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.827425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:57:24.827734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.829364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.841193 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1427/2816] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:57:24.845350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.855006 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66eb181-2536-47b8-aaa7-332ea7a3514c None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.859650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.860284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.860595 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.860864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.861553 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.863423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.893117 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1390/2817] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:24 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:57:24.897757 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:24.908060 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb627b64-57e8-4884-b0e2-e5c18617b988 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:24.934475 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:24.965675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:24.966304 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:24.966584 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:24.966856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:24.967535 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:24.969498 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:24.990225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1428/2818] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:24 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:24.994767 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.005159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9d784-7054-48e9-b66a-8f6be965e40a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.009762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.010410 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.010688 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.010963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.011686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.013344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.024067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1391/2819] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:25.027807 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.038988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c39513-5237-43f9-a86f-dcc2ed2c8f11 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.043965 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.044653 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.044937 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.045214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.045943 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.047523 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202848b5-e570-4722-a7ba-0a24257caad2 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.057326 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1429/2820] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:25.061311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.071474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f78f62-4671-4aa9-9b3e-5f54f106d36a admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.076287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.077202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.077491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.077766 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] PATH_INFO: `/v3/projects/eda1e4a9decb45fda421c46f2a0f6b6f/users/1bd6aefeaab14aceb58249f4e0419902/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.092932 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eda1e4a9decb45fda421c46f2a0f6b6f, user_id=1bd6aefeaab14aceb58249f4e0419902, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.095641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.109323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1392/2821] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:25 2022] PUT /identity/v3/projects/eda1e4a9decb45fda421c46f2a0f6b6f/users/1bd6aefeaab14aceb58249f4e0419902/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:57:25.120341 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.120636 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.120915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.147352 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] MFA Rules not processed for user `1bd6aefeaab14aceb58249f4e0419902`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:25.148628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:25.214458 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1430/2822] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:25.218924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.281807 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df60623a-f11c-4f9e-9334-e55942b82f65 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.286441 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.287092 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.287369 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.287641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.288341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.290263 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.318596 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1393/2823] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:25 2022] POST /identity/v3/projects => generated 401 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:25.319741 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.330602 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac5791-7a15-476d-b3a9-1fad2efbaf56 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.387380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.388049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.388355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.388634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.389331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.391240 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.410298 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1431/2824] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:25 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:57:25.414773 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.425038 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280e2ca-37a5-4492-916c-a2881397b48c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.430029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.430677 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.430960 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.431235 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.431950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.433590 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.443772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1394/2825] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:25.447768 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.457626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6190cb-3311-492b-b253-a4b87e9f5cdd admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.462264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.463139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.463422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.463693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] PATH_INFO: `/v3/projects/54f8ed22ab264994b4f9ac74499a02c5/users/0b682ce618754deeabacfe79e87f8f5b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.478881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f8ed22ab264994b4f9ac74499a02c5, user_id=0b682ce618754deeabacfe79e87f8f5b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.481502 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.495409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1432/2826] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:25 2022] PUT /identity/v3/projects/54f8ed22ab264994b4f9ac74499a02c5/users/0b682ce618754deeabacfe79e87f8f5b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:25.499153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.508978 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84123e4-7cd5-4846-aa5a-06a2f3417885 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.528055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:25.565383 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.566039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.566319 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.566622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.567364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.569003 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc17c9-4e7c-4850-86ab-0a08dcd2be49 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.578451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1395/2827] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:25.582493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.588418 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:25.594141 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c335-c70d-4507-b645-4250d5582489 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.600509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.601451 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.601955 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.602226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] PATH_INFO: `/v3/projects/54f8ed22ab264994b4f9ac74499a02c5/users/0b682ce618754deeabacfe79e87f8f5b/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.604170 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54f8ed22ab264994b4f9ac74499a02c5, user_id=0b682ce618754deeabacfe79e87f8f5b, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.606838 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.616855 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1433/2828] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:25 2022] PUT /identity/v3/projects/54f8ed22ab264994b4f9ac74499a02c5/users/0b682ce618754deeabacfe79e87f8f5b/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:25.627122 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.627609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.627694 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.653715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] MFA Rules not processed for user `0b682ce618754deeabacfe79e87f8f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:25.654901 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:25.722353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1396/2829] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:25.732957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.734687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:25.790874 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cf2f6-7ac6-4f6d-88fc-e5ea00080fe7 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.794510 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.795149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.795459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.795739 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.805124 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.806555 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.815822 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1434/2830] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:25.892478 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:25.902571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d117bb-46fd-4cd1-8ec8-fd63ce8f7642 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:25.906166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:25.906799 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:25.907080 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:25.907357 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:25.963672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:25.965146 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbea6382-c53d-4e7c-ba20-073fc4e9d425 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:25.973867 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1397/2831] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:30.026135 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:30.089044 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e23f2a-34e7-4560-9032-391d9bc8ae7b service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:30.093511 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:30.094379 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:30.094662 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:30.094935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] PATH_INFO: `/v3/users/1bd6aefeaab14aceb58249f4e0419902` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:30.096109 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bd6aefeaab14aceb58249f4e0419902)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:30.098865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:30.143177 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bd6aefeaab14aceb58249f4e0419902'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.143845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bd6aefeaab14aceb58249f4e0419902'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.144484 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bd6aefeaab14aceb58249f4e0419902'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.155606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bd6aefeaab14aceb58249f4e0419902'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.163122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bd6aefeaab14aceb58249f4e0419902'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.179816 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1435/2832] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:30 2022] DELETE /identity/v3/users/1bd6aefeaab14aceb58249f4e0419902 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:30.184271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:30.184762 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:30.184862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:30.201575 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:30.202759 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:30.251976 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=100 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:30.274723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1398/2833] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:30.284246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:30.286367 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:30.299705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:30.345668 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0343da9d-7b64-4ab9-8cc3-67b19f7099aa admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:30.350039 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:30.350704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:30.351025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:30.351338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:30.360886 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:30.362703 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:30.373154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1436/2834] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:30.635826 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:30.637290 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:30.649687 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412581f0-93a6-44b6-926c-5f8fb5dd72a3 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:30.654519 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:30.655377 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:30.655667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:30.655925 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] PATH_INFO: `/v3/projects/eda1e4a9decb45fda421c46f2a0f6b6f` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:30.657135 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eda1e4a9decb45fda421c46f2a0f6b6f)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:30.659083 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:30.706137 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eda1e4a9decb45fda421c46f2a0f6b6f', 'request_id': 'req-bbaf1e2b-cb9c-415f-b86b-824369330857'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.712161 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eda1e4a9decb45fda421c46f2a0f6b6f', 'request_id': 'req-bbaf1e2b-cb9c-415f-b86b-824369330857'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.719165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eda1e4a9decb45fda421c46f2a0f6b6f', 'request_id': 'req-bbaf1e2b-cb9c-415f-b86b-824369330857'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.730009 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] The token cache is being invalidate because project eda1e4a9decb45fda421c46f2a0f6b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:30.730291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.730907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.732496 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1399/2835] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:30 2022] DELETE /identity/v3/projects/eda1e4a9decb45fda421c46f2a0f6b6f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:30.736772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:30.747437 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6bdf3-938b-4ca7-9d97-b4aaf9bd3100 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:30.807281 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:30.808088 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:30.808405 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:30.808686 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] PATH_INFO: `/v3/users/0b682ce618754deeabacfe79e87f8f5b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:30.809883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b682ce618754deeabacfe79e87f8f5b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:30.811792 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:30.856744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b682ce618754deeabacfe79e87f8f5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.857406 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b682ce618754deeabacfe79e87f8f5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.858016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b682ce618754deeabacfe79e87f8f5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.865856 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b682ce618754deeabacfe79e87f8f5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.874061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-27685271-1a5f-4eb3-badb-89bce5fe59e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b682ce618754deeabacfe79e87f8f5b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:30.891549 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1437/2836] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:30 2022] DELETE /identity/v3/users/0b682ce618754deeabacfe79e87f8f5b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:31.172388 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.173931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:31.194505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf1e2b-cb9c-415f-b86b-824369330857 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.256370 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.257555 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.257842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.258118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] PATH_INFO: `/v3/projects/54f8ed22ab264994b4f9ac74499a02c5` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.259300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54f8ed22ab264994b4f9ac74499a02c5)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.261309 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.308873 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54f8ed22ab264994b4f9ac74499a02c5', 'request_id': 'req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:31.314469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54f8ed22ab264994b4f9ac74499a02c5', 'request_id': 'req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:31.321618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54f8ed22ab264994b4f9ac74499a02c5', 'request_id': 'req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:31.328549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:31.333695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] The token cache is being invalidate because project 54f8ed22ab264994b4f9ac74499a02c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:31.334047 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:31.334769 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-116d65ae-bfe7-4502-b7b2-ac4273ccd0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:31.336845 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1400/2837] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:31 2022] DELETE /identity/v3/projects/54f8ed22ab264994b4f9ac74499a02c5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:31.341729 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.342176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.342262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.360954 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:31.362165 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:31.393397 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:31.438380 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1438/2838] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:57:31.446044 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.446341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.446622 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.465345 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:31.466715 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:31.476279 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1401/2839] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:31.480488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.490851 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b16f09-9da8-4c68-ba35-6ca86def553c None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.495675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.496348 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.496632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.496903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.497627 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80366) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:57:31.497931 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.499516 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.510834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1439/2840] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:57:31.514951 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.524712 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59107b5-fa08-4abf-a33e-1798e2e8aa3b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.529402 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.530046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.530328 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.530602 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.531324 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.533315 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.559096 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1402/2841] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:31 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:31.563211 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.573639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7b04d-f61f-4393-8228-b1a0fa20aca4 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.629670 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.630330 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.630610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.630880 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] PATH_INFO: `/v3/users` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.631557 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.633483 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.653032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1440/2842] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:31 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:31.657071 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.667850 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c660f2d-2154-41c0-b2c7-613f3048aef6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.672376 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.673023 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.673300 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.673569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.674287 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.675880 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.685929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1403/2843] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:31.689504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.699413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed3270-bce7-4a12-893b-5b60ef74e904 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.703970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.704652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.704934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.705209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.705929 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.707998 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93843e6-205f-4a75-9bec-5fd0a29482ac admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.717673 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1441/2844] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:57:31.721486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.732104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecb4f0-5c3e-4ede-8446-5984acabf1f3 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.736647 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.737502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.737778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.738043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] PATH_INFO: `/v3/projects/d3584cd531fd45bb8e9124ac23bc3923/users/a0f2f9cfd6624578b5617ec93119c388/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.752731 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3584cd531fd45bb8e9124ac23bc3923, user_id=a0f2f9cfd6624578b5617ec93119c388, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.755390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.767981 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1404/2845] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:31 2022] PUT /identity/v3/projects/d3584cd531fd45bb8e9124ac23bc3923/users/a0f2f9cfd6624578b5617ec93119c388/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:31.773615 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c12c-db16-4533-9231-6817c603b4af None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.773902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c12c-db16-4533-9231-6817c603b4af None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.774174 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0366c12c-db16-4533-9231-6817c603b4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.792685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0366c12c-db16-4533-9231-6817c603b4af None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:31.793863 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0366c12c-db16-4533-9231-6817c603b4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:31.851789 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1442/2846] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:31.861198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:31.862878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:31.919754 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577b88d-c34c-4420-ac87-283d685f6ba5 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:31.923355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:31.924018 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:31.924311 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:31.924598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:31.933750 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:31.935185 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:31.944921 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1405/2847] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:32.902192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:32.903079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:32.903569 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:32.919342 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:32.928923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9700-db98-4884-8c08-9cf746a15ed4 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:32.932663 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:32.933210 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] MFA Rules not processed for user `a0f2f9cfd6624578b5617ec93119c388`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:32.933323 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:32.933583 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:32.933859 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:32.934492 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:32.935629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:32.966090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:33.010907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.012047 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1443/2848] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 15:57:33.012915 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.016006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.027515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf1870-09b7-4184-b40e-7d5738198ac8 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.032801 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1406/2849] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:57:33.033292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.033982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.034269 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.034546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.036120 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.039042 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.075340 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1444/2850] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:33 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:57:33.078970 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.092699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eeea4-76bf-4119-b27c-d77b73843277 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.123680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:33.153862 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.154630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.155411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.155411 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.155878 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.157907 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.184797 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1407/2851] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:33 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:57:33.194438 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.205366 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d04e8-7d99-4887-a9fc-88e6a923a370 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.210819 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.211509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.211794 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.212071 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.212853 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.214467 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.226875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1445/2852] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:33.231165 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.241299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c40c8-2bf6-45a2-bb20-72f6a2838ef6 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.246699 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.247653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.247952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.248221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] PATH_INFO: `/v3/projects/97f5c3ac78d3485492407f93b19386a3/users/cf2bb9ccac2041a09754a6e3e536873e/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.264194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f5c3ac78d3485492407f93b19386a3, user_id=cf2bb9ccac2041a09754a6e3e536873e, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.266924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.281016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1408/2853] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:33 2022] PUT /identity/v3/projects/97f5c3ac78d3485492407f93b19386a3/users/cf2bb9ccac2041a09754a6e3e536873e/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:33.285413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.296219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c10cec-2172-4a96-b4fc-d733d4d9031c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.357145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.357866 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.358147 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.358417 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.359138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.360782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe38ab1-f69d-47f5-b801-d61e66608454 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.373918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1446/2854] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:33.375254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:33.385392 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16544bc-9416-4743-98ec-497d40892e30 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:33.390026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:33.390889 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:33.391166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:33.391437 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] PATH_INFO: `/v3/projects/97f5c3ac78d3485492407f93b19386a3/users/cf2bb9ccac2041a09754a6e3e536873e/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:33.393412 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97f5c3ac78d3485492407f93b19386a3, user_id=cf2bb9ccac2041a09754a6e3e536873e, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:33.396578 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:33.406955 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1409/2855] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:33 2022] PUT /identity/v3/projects/97f5c3ac78d3485492407f93b19386a3/users/cf2bb9ccac2041a09754a6e3e536873e/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:57:34.041486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:34.041986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:34.041986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:34.073087 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] MFA Rules not processed for user `cf2bb9ccac2041a09754a6e3e536873e`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:34.074477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:34.153719 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1447/2856] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:34.166202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:34.168159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:34.246707 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82de86-59e4-403d-9577-122ed80aae34 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:34.250486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:34.251141 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:34.251421 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:34.251697 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:34.310709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:34.312151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:34.321675 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1410/2857] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:35.672037 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:35.673509 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:35.685820 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71c955-ff71-4dc3-8dfe-c3f701e95ee6 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:35.689355 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:35.690030 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:35.690312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:35.690588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:35.699449 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:35.700935 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:35.710707 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1448/2858] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:44.318111 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:45.706262 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:48.515139 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:48.579792 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6293b-f24a-4cdc-af29-5e539f169b89 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:48.584902 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:48.585772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:48.586055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:48.586332 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] PATH_INFO: `/v3/users/a0f2f9cfd6624578b5617ec93119c388` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:48.587512 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f2f9cfd6624578b5617ec93119c388)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:48.589500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:48.634569 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f2f9cfd6624578b5617ec93119c388'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.635259 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f2f9cfd6624578b5617ec93119c388'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.635916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f2f9cfd6624578b5617ec93119c388'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.642968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f2f9cfd6624578b5617ec93119c388'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.655434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f2f9cfd6624578b5617ec93119c388'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.668626 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1411/2859] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:48 2022] DELETE /identity/v3/users/a0f2f9cfd6624578b5617ec93119c388 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:48.832585 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:48.843331 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9ed11-badd-4d67-b083-98315a05af96 service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:48.907882 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:48.908796 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:48.909081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:48.909359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] PATH_INFO: `/v3/projects/d3584cd531fd45bb8e9124ac23bc3923` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:48.910562 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3584cd531fd45bb8e9124ac23bc3923)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:48.912532 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:48.958303 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3584cd531fd45bb8e9124ac23bc3923', 'request_id': 'req-875086d5-a0e8-4060-bcdb-47e584a2edc6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.963873 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3584cd531fd45bb8e9124ac23bc3923', 'request_id': 'req-875086d5-a0e8-4060-bcdb-47e584a2edc6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.970865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3584cd531fd45bb8e9124ac23bc3923', 'request_id': 'req-875086d5-a0e8-4060-bcdb-47e584a2edc6'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.981616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] The token cache is being invalidate because project d3584cd531fd45bb8e9124ac23bc3923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:48.981896 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.982518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:48.984040 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1449/2860] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:48 2022] DELETE /identity/v3/projects/d3584cd531fd45bb8e9124ac23bc3923 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:57:48.987952 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:48.989860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:49.003120 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427004-5da8-40b6-9bf9-4c4b2e3cb750 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.060456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.061317 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.061599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.061872 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] PATH_INFO: `/v3/users/cf2bb9ccac2041a09754a6e3e536873e` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.063067 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2bb9ccac2041a09754a6e3e536873e)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.065070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.087355 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:49.112579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2bb9ccac2041a09754a6e3e536873e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.113232 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2bb9ccac2041a09754a6e3e536873e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.113839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2bb9ccac2041a09754a6e3e536873e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.121021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2bb9ccac2041a09754a6e3e536873e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.132741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8abc6bda-20b2-400e-a7a2-5db3e95621fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2bb9ccac2041a09754a6e3e536873e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.144778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1412/2861] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:48 2022] DELETE /identity/v3/users/cf2bb9ccac2041a09754a6e3e536873e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:49.326788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.328224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:49.342607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875086d5-a0e8-4060-bcdb-47e584a2edc6 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.399436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.400319 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.400606 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.400875 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] PATH_INFO: `/v3/projects/97f5c3ac78d3485492407f93b19386a3` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.402028 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97f5c3ac78d3485492407f93b19386a3)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.403945 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.437270 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:49.450716 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97f5c3ac78d3485492407f93b19386a3', 'request_id': 'req-f1731c32-4a75-40e6-8b27-3e38d905f445'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.456186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97f5c3ac78d3485492407f93b19386a3', 'request_id': 'req-f1731c32-4a75-40e6-8b27-3e38d905f445'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.463934 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97f5c3ac78d3485492407f93b19386a3', 'request_id': 'req-f1731c32-4a75-40e6-8b27-3e38d905f445'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.475009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] The token cache is being invalidate because project 97f5c3ac78d3485492407f93b19386a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:57:49.475292 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.475922 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-f1731c32-4a75-40e6-8b27-3e38d905f445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:49.477486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1450/2862] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:57:49 2022] DELETE /identity/v3/projects/97f5c3ac78d3485492407f93b19386a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:49.482276 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.482564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.482838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.500200 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:49.501341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:49.562770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1413/2863] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:49.572054 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.572371 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.572676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.590654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:49.591828 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:49.599704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1451/2864] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:49.603571 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.613045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda712c-84b7-4deb-9dbd-742af488e85b None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.618344 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.618983 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.619261 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.619531 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.620240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:57:49.620576 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.622172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.633942 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1414/2865] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:57:49.637971 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.647003 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39d5d2-a693-44e0-9d38-2a04c5fcb4c5 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.651663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.652322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.652612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.652884 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.653623 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.655546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.680907 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1452/2866] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:57:49.684159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.694539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482bcce-a6b0-4b96-8a60-116a8f53bb90 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.750536 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.751156 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.751445 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.751727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.752468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.755073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.774899 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1415/2867] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:49.779034 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.789143 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438cf4b-e0d5-450a-a361-9da264c1083e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.793674 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.794329 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.794610 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.794883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.795625 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.797241 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.807927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1453/2868] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:49.811594 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.822021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fe001-25aa-4d69-aa37-bdd8414315bc admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.826629 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.827271 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.827548 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.827819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.828561 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.830151 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca241ee-e840-4d4c-9664-290aec76e0e7 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.839777 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1416/2869] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:49.843607 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:49.853284 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44cfc2-8d09-4c03-84e1-fe4423c5544e admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:49.857837 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:49.858711 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.858991 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.859266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] PATH_INFO: `/v3/projects/1ee6c80114aa436dbf2fcdeb4eca7438/users/aa5368e58ff84ac4a0c24f6398d12562/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.874235 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee6c80114aa436dbf2fcdeb4eca7438, user_id=aa5368e58ff84ac4a0c24f6398d12562, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:49.876902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:49.889771 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1454/2870] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:49 2022] PUT /identity/v3/projects/1ee6c80114aa436dbf2fcdeb4eca7438/users/aa5368e58ff84ac4a0c24f6398d12562/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 15:57:49.899881 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:49.900177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:49.900489 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:49.906995 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:49.928722 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] MFA Rules not processed for user `aa5368e58ff84ac4a0c24f6398d12562`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:49.929974 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:49.993124 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1417/2871] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:49.997540 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.059372 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13928b4e-f3f8-427b-ab25-b9d04bda083b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.063840 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.064553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.064834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.065102 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.065780 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.067705 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.093577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1455/2872] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:57:49 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:50.097564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.108651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ec443-4c53-453c-8834-e4551ce906c4 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.163770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.164450 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.164739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.165010 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.165680 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.167592 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.187739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1418/2873] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:57:50 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:57:50.191652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.202078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916be45-a323-4c17-bac9-0338cc87fedf admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.206638 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.207287 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.207666 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.207940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.208701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.210654 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.221399 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1456/2874] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:50.226178 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.236164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688da0b-5cb0-4830-a3b2-557a8737d3da admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.241439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.242308 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.242846 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.243129 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] PATH_INFO: `/v3/projects/1c2741aada7a4259a836fb04dc96229e/users/a58e811f71514775a4be94677e7218cf/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.257474 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2741aada7a4259a836fb04dc96229e, user_id=a58e811f71514775a4be94677e7218cf, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.260172 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.273508 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1419/2875] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:50 2022] PUT /identity/v3/projects/1c2741aada7a4259a836fb04dc96229e/users/a58e811f71514775a4be94677e7218cf/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:50.277286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.288000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d08b1-7e3c-4252-956c-9aa06664a865 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.316025 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:50.344550 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.345209 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.345488 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.345758 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.346480 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.348058 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b733-6e08-4abb-bd6b-e21b8e0ff8c6 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.358132 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1457/2876] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:57:50.362186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.372621 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20ac0-d948-4d78-85f0-801705a82a9f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.377211 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.378082 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.378363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.378638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] PATH_INFO: `/v3/projects/1c2741aada7a4259a836fb04dc96229e/users/a58e811f71514775a4be94677e7218cf/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.380627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2741aada7a4259a836fb04dc96229e, user_id=a58e811f71514775a4be94677e7218cf, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.383260 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.393297 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1420/2877] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:57:50 2022] PUT /identity/v3/projects/1c2741aada7a4259a836fb04dc96229e/users/a58e811f71514775a4be94677e7218cf/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:50.403512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.403801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.404078 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.430997 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] MFA Rules not processed for user `a58e811f71514775a4be94677e7218cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:50.432239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:50.498307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1458/2878] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:50.508668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:50.510341 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:50.561896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:50.568265 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09828d-4365-4f65-b7dc-4f95009b120f admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:50.571957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:50.572623 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:50.572903 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:50.573177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:50.628039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:50.629639 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:50.638459 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1421/2879] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:54.760685 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:54.762107 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:54.774186 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a7af7-e52b-4c14-869f-f0db0234811d None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:54.777747 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:54.778374 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:54.778652 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:54.778919 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:54.788232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:54.789677 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe820a5b-3e3f-47f4-91d2-bcbf2a3a1820 service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:54.798872 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1459/2880] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:54 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:57:59.625159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:59.686254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c561642-4031-495f-bb4a-6f6c4c6b911f service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:59.690752 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:59.691603 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:59.692400 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:59.692672 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] PATH_INFO: `/v3/users/aa5368e58ff84ac4a0c24f6398d12562` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:59.693822 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa5368e58ff84ac4a0c24f6398d12562)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:59.695737 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:59.740505 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa5368e58ff84ac4a0c24f6398d12562'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:59.741150 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa5368e58ff84ac4a0c24f6398d12562'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:59.741751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa5368e58ff84ac4a0c24f6398d12562'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:59.748886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa5368e58ff84ac4a0c24f6398d12562'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:59.761068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa5368e58ff84ac4a0c24f6398d12562'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:57:59.774227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1422/2881] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:57:59 2022] DELETE /identity/v3/users/aa5368e58ff84ac4a0c24f6398d12562 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:57:59.778647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:59.779156 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:59.779248 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:59.797112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:57:59.798286 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:59.855274 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1460/2882] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:57:59.864755 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:57:59.866153 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:57:59.888491 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:57:59.926064 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a913-9e1f-4f4b-86ab-f67b5486cfbe admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:57:59.929651 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:57:59.930289 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:57:59.930564 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:57:59.930834 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:57:59.940434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:57:59.942390 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:57:59.951730 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1423/2883] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:57:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:00.216180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:00.217663 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:00.238009 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019736c-f678-47fa-ac6c-3533d2f99487 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:00.244000 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:00.245236 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:00.245518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:00.245850 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] PATH_INFO: `/v3/projects/1ee6c80114aa436dbf2fcdeb4eca7438` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:00.247090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ee6c80114aa436dbf2fcdeb4eca7438)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:00.249056 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:00.285785 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:00.296057 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ee6c80114aa436dbf2fcdeb4eca7438', 'request_id': 'req-be364804-4ae4-4cc3-8e40-61221c8cc13b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.301891 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ee6c80114aa436dbf2fcdeb4eca7438', 'request_id': 'req-be364804-4ae4-4cc3-8e40-61221c8cc13b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.308783 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ee6c80114aa436dbf2fcdeb4eca7438', 'request_id': 'req-be364804-4ae4-4cc3-8e40-61221c8cc13b'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.320387 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] The token cache is being invalidate because project 1ee6c80114aa436dbf2fcdeb4eca7438 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:58:00.320704 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.321560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.323096 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1461/2884] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:58:00 2022] DELETE /identity/v3/projects/1ee6c80114aa436dbf2fcdeb4eca7438 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:00.327039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:00.336522 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f97ee1-487e-4353-843c-e73ec79538fb service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:00.391242 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:00.392043 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:00.392356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:00.392638 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] PATH_INFO: `/v3/users/a58e811f71514775a4be94677e7218cf` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:00.393809 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58e811f71514775a4be94677e7218cf)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:00.395789 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:00.441761 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58e811f71514775a4be94677e7218cf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.442384 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58e811f71514775a4be94677e7218cf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.442977 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58e811f71514775a4be94677e7218cf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.449821 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58e811f71514775a4be94677e7218cf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.461470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-f3646e32-a307-43d3-830c-3649035f4532 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58e811f71514775a4be94677e7218cf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.474026 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1424/2885] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:58:00 2022] DELETE /identity/v3/users/a58e811f71514775a4be94677e7218cf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:00.735178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:00.736804 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:00.752266 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be364804-4ae4-4cc3-8e40-61221c8cc13b admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:00.808980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:00.809849 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:00.810133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:00.810409 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] PATH_INFO: `/v3/projects/1c2741aada7a4259a836fb04dc96229e` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:00.811586 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2741aada7a4259a836fb04dc96229e)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:00.813506 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:00.858692 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2741aada7a4259a836fb04dc96229e', 'request_id': 'req-609a9dd4-dfec-4d30-87de-9a1427f005fa'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.864121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2741aada7a4259a836fb04dc96229e', 'request_id': 'req-609a9dd4-dfec-4d30-87de-9a1427f005fa'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.871358 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2741aada7a4259a836fb04dc96229e', 'request_id': 'req-609a9dd4-dfec-4d30-87de-9a1427f005fa'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.882128 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] The token cache is being invalidate because project 1c2741aada7a4259a836fb04dc96229e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:58:00.882412 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.883029 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-609a9dd4-dfec-4d30-87de-9a1427f005fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:00.884573 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1462/2886] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:58:00 2022] DELETE /identity/v3/projects/1c2741aada7a4259a836fb04dc96229e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:00.890486 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:00.890924 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:00.891031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:00.908732 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:00.909858 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:00.948187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:00.970609 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1425/2887] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:00.980629 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:00.980914 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:00.981196 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:00.999476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:01.000701 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:01.008558 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1463/2888] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:01.012842 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.023349 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f9be7d-8ea6-4353-9376-2d9a8d14b698 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.027816 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.028604 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.028888 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.029159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.029882 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80365) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 15:58:01.030253 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.031861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.045224 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1426/2889] 10.222.0.45 () {58 vars in 1152 bytes} [Tue May 24 15:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 15:58:01.049552 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.058647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddb295-2815-46da-a4a6-57827dde13c9 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.063275 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.063920 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.064207 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.064515 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.065231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.067171 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.094338 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1464/2890] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:58:01.098963 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.108743 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44e2e16-9868-417b-ba84-cbea284a3d6c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.165070 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.165916 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.166352 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.166685 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.167480 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.169539 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.188187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1427/2891] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:58:01.192918 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.203005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5723d-610a-473f-94b5-db482d019458 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.208176 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.208894 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.209181 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.209456 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.210178 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.211767 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.222112 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1465/2892] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:58:01.225923 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.236435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcada5f3-11d0-48f4-bb13-6a28ed52da2c admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.240971 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.241643 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.241922 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.242209 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.243164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.244791 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae04ab-991c-44d0-b0c5-9fa3f2e09f2c admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.255202 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1428/2893] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:58:01.259179 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.269150 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70492a29-e646-4aa5-b241-1ae6ab3acf32 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.274343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.275222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.275504 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.275782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] PATH_INFO: `/v3/projects/36a6d1ffb28f4c1b85db07eb4bdc66c8/users/10a2b4f0948c4470bea5e782316d7914/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.287772 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:01.293090 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a6d1ffb28f4c1b85db07eb4bdc66c8, user_id=10a2b4f0948c4470bea5e782316d7914, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.296079 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.308969 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1466/2894] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:58:01 2022] PUT /identity/v3/projects/36a6d1ffb28f4c1b85db07eb4bdc66c8/users/10a2b4f0948c4470bea5e782316d7914/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:01.320468 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.320756 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.321029 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.347957 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] MFA Rules not processed for user `10a2b4f0948c4470bea5e782316d7914`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:01.349171 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:01.411537 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1429/2895] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:01.415982 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.480125 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26201b6f-382e-40a7-80b4-ebe2175ffb65 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.484940 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.485603 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.485883 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.486154 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.486889 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.488865 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.514005 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1467/2896] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:58:01.518022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.529500 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ad88-146d-458b-be49-4daaf2276662 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.563046 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:01.587189 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.587860 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.588142 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.588455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.589163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.591118 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.611208 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1430/2897] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 15:58:01.615751 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.626297 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6dcb3c-0bc7-4f5d-8600-3aef543a499a admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.631428 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.632081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.632398 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.632676 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.633401 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.634980 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.645271 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1468/2898] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:58:01.649027 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.658673 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e54bb3-fc5f-495c-bf8f-5824f6b5b99b admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.663299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.663944 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.664227 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.664532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] PATH_INFO: `/v3/roles` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.665254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.666896 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75329127-29e2-4783-b208-ca87d9f7108d admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.676950 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1431/2899] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 15:58:01.680678 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.690259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6229d-1032-46ea-9b33-526b4912acf1 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.695378 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.696232 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.696534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.696801 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] PATH_INFO: `/v3/projects/8c7e6e2b16774cc995dee200f730a8a9/users/8052c556433b433fb947184d8c409a38/roles/82e7d694fe0846dda33cc001bd87aa4a` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.711121 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7e6e2b16774cc995dee200f730a8a9, user_id=8052c556433b433fb947184d8c409a38, role_id=82e7d694fe0846dda33cc001bd87aa4a)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.713774 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.726346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1469/2900] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:58:01 2022] PUT /identity/v3/projects/8c7e6e2b16774cc995dee200f730a8a9/users/8052c556433b433fb947184d8c409a38/roles/82e7d694fe0846dda33cc001bd87aa4a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:01.737159 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ba38-d211-4840-b427-078edb833d92 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.737476 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ba38-d211-4840-b427-078edb833d92 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.737757 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820ba38-d211-4840-b427-078edb833d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.764781 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-7820ba38-d211-4840-b427-078edb833d92 None None] MFA Rules not processed for user `8052c556433b433fb947184d8c409a38`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:01.765973 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7820ba38-d211-4840-b427-078edb833d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:01.828155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1432/2901] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:01.832505 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.892690 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f186e33c-18bf-4454-9095-a9f237728c5c admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.897278 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.897927 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.898295 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.898469 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] PATH_INFO: `/v3/projects` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.899139 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:01.901062 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:01.925759 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1470/2902] 10.222.0.45 () {60 vars in 1152 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:58:01.930275 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820ba38-d211-4840-b427-078edb833d92 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:01.940853 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820ba38-d211-4840-b427-078edb833d92 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:01.996246 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:01.996939 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:01.997221 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:01.997498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] PATH_INFO: `/v3/users` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:01.998186 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.000104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.019176 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1433/2903] 10.222.0.45 () {60 vars in 1143 bytes} [Tue May 24 15:58:01 2022] POST /identity/v3/users => generated 299 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 15:58:02.023336 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:02.033643 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a09f5b-2780-4b0c-8e4f-75687e7f659f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:02.039138 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:02.039948 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.040220 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.040583 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.041308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.042903 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.054802 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1471/2904] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:58:02.058836 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:02.068772 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057585d-6ba5-475d-a3d2-db7644985caa admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:02.073458 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:02.074336 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.074618 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.074895 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] PATH_INFO: `/v3/projects/6714c4681f0a4751bf63e59f55b45b05/users/cf1fd0d089e34556b6f6a90b6f5c74d4/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.089691 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6714c4681f0a4751bf63e59f55b45b05, user_id=cf1fd0d089e34556b6f6a90b6f5c74d4, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.092365 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.105353 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1434/2905] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:58:02 2022] PUT /identity/v3/projects/6714c4681f0a4751bf63e59f55b45b05/users/cf1fd0d089e34556b6f6a90b6f5c74d4/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:02.109224 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:02.119798 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ae37f-a1ec-4ebf-b3e1-275951213456 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:02.128514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:02.175206 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:02.175909 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.176744 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.177016 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.177728 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.179308 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd18b0b-70c0-49e2-9a8b-38bc59f79b43 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.188634 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1472/2906] 10.222.0.45 () {58 vars in 1121 bytes} [Tue May 24 15:58:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 15:58:02.192700 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:02.202886 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855daef-a3ee-4589-8e4b-b805a59df939 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:02.207466 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:02.208378 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.208693 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.208969 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] PATH_INFO: `/v3/projects/6714c4681f0a4751bf63e59f55b45b05/users/cf1fd0d089e34556b6f6a90b6f5c74d4/roles/6dfe90fc0a2441a881142526cd3c7974` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.210914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6714c4681f0a4751bf63e59f55b45b05, user_id=cf1fd0d089e34556b6f6a90b6f5c74d4, role_id=6dfe90fc0a2441a881142526cd3c7974)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.213606 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.219075 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:02.225532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1435/2907] 10.222.0.45 () {60 vars in 1482 bytes} [Tue May 24 15:58:02 2022] PUT /identity/v3/projects/6714c4681f0a4751bf63e59f55b45b05/users/cf1fd0d089e34556b6f6a90b6f5c74d4/roles/6dfe90fc0a2441a881142526cd3c7974 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:02.235546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.235857 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.236133 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.263122 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] MFA Rules not processed for user `cf1fd0d089e34556b6f6a90b6f5c74d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:02.264364 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:02.329592 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1473/2908] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:02.341558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:02.343237 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:02.400779 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ad17-e3c6-446b-aa6a-154f6f543f76 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:02.404347 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:02.405053 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:02.405331 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:02.405600 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:02.459637 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:02.461055 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:02.469684 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1436/2909] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:58:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:03.947472 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:03.948890 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:03.961032 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78254c6-0911-4307-b10a-c042b9b9d278 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:03.964548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:03.965187 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:03.965465 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:03.965737 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:04.022413 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:04.023868 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:04.032797 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1474/2910] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:58:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:04.571824 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:04.580415 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f9771-18d4-4788-918f-590dcb345ef8 service cinder] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:04.584073 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '94ce1868b4ac4d2a8273867aab1bfe80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:04.584739 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:04.585021 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:04.585295 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:04.594254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:04.595695 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e5309-73bf-4ae5-893d-b240311f40c9 service cinder] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:04.605136 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1437/2911] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:58:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:08.423787 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:08.457669 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=100 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:08.486246 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df005c61-e080-4c55-b4f7-feb30a4a91ed service cinder] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:08.490753 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:08.491626 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:08.491910 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:08.492470 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] PATH_INFO: `/v3/users/10a2b4f0948c4470bea5e782316d7914` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:08.493924 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a2b4f0948c4470bea5e782316d7914)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:08.495885 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:08.543346 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a2b4f0948c4470bea5e782316d7914'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:08.543994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a2b4f0948c4470bea5e782316d7914'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:08.544616 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a2b4f0948c4470bea5e782316d7914'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:08.551598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a2b4f0948c4470bea5e782316d7914'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:08.558542 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a2b4f0948c4470bea5e782316d7914'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:08.575216 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1475/2912] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:58:08 2022] DELETE /identity/v3/users/10a2b4f0948c4470bea5e782316d7914 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:08.579633 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:08.580104 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:08.580201 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:08.597198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] MFA Rules not processed for user `3a944101ee294a37aa42cee1133a9032`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:58:08.599045 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:08.656059 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1438/2913] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:58:08.666015 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:08.667416 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:08.724582 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255c281-383e-4c31-8ac8-9118baced0cf admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:08.728180 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:08.728881 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:08.729167 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:08.729444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:08.738362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:08.739793 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:08.749459 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1476/2914] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:58:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:09.015270 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:09.016751 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:09.029498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab343b-7b23-4f9f-94c5-13510a5a0a39 None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:09.034163 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:09.035035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:09.035306 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:09.035579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] PATH_INFO: `/v3/projects/36a6d1ffb28f4c1b85db07eb4bdc66c8` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:09.036813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a6d1ffb28f4c1b85db07eb4bdc66c8)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:09.038784 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:09.076746 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:09.087364 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a6d1ffb28f4c1b85db07eb4bdc66c8', 'request_id': 'req-960ec50b-9167-45ee-88de-7239e53a3aaf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.092798 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a6d1ffb28f4c1b85db07eb4bdc66c8', 'request_id': 'req-960ec50b-9167-45ee-88de-7239e53a3aaf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.100433 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a6d1ffb28f4c1b85db07eb4bdc66c8', 'request_id': 'req-960ec50b-9167-45ee-88de-7239e53a3aaf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.111461 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] The token cache is being invalidate because project 36a6d1ffb28f4c1b85db07eb4bdc66c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:58:09.111741 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.112405 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.114418 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1439/2915] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:58:09 2022] DELETE /identity/v3/projects/36a6d1ffb28f4c1b85db07eb4bdc66c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:09.118548 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:09.127986 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc00010-1e3d-4bef-9fa7-16937243aaad service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:09.176594 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:09.185382 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:09.186318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:09.186693 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:09.187027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] PATH_INFO: `/v3/users/8052c556433b433fb947184d8c409a38` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:09.188512 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8052c556433b433fb947184d8c409a38)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:09.190444 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:09.235157 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8052c556433b433fb947184d8c409a38'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.235790 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8052c556433b433fb947184d8c409a38'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.236477 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8052c556433b433fb947184d8c409a38'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.244244 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8052c556433b433fb947184d8c409a38'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.251008 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8052c556433b433fb947184d8c409a38'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.268083 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1477/2916] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:58:09 2022] DELETE /identity/v3/users/8052c556433b433fb947184d8c409a38 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:09.525686 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:09.527245 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:09.543198 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ec50b-9167-45ee-88de-7239e53a3aaf admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:09.601101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:09.601968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:09.602250 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:09.602525 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] PATH_INFO: `/v3/projects/8c7e6e2b16774cc995dee200f730a8a9` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:09.603689 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7e6e2b16774cc995dee200f730a8a9)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:09.605630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:09.650671 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7e6e2b16774cc995dee200f730a8a9', 'request_id': 'req-8a094b34-245f-4b05-9af7-c45da94820a1'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.656039 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7e6e2b16774cc995dee200f730a8a9', 'request_id': 'req-8a094b34-245f-4b05-9af7-c45da94820a1'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.664552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7e6e2b16774cc995dee200f730a8a9', 'request_id': 'req-8a094b34-245f-4b05-9af7-c45da94820a1'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.675434 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] The token cache is being invalidate because project 8c7e6e2b16774cc995dee200f730a8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:58:09.675719 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.676911 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.678423 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1440/2917] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:58:09 2022] DELETE /identity/v3/projects/8c7e6e2b16774cc995dee200f730a8a9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:09.682342 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:09.683734 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:09.696104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb38d366-ccee-4f9c-b4cd-873eaf02dd23 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:09.750970 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:09.751834 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:09.752115 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:09.752423 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] PATH_INFO: `/v3/users/cf1fd0d089e34556b6f6a90b6f5c74d4` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:09.753577 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf1fd0d089e34556b6f6a90b6f5c74d4)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:09.755541 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:09.800544 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf1fd0d089e34556b6f6a90b6f5c74d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.801182 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf1fd0d089e34556b6f6a90b6f5c74d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.801786 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf1fd0d089e34556b6f6a90b6f5c74d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.808645 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf1fd0d089e34556b6f6a90b6f5c74d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.815306 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf1fd0d089e34556b6f6a90b6f5c74d4'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:09.825247 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:09.834628 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1478/2918] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 15:58:09 2022] DELETE /identity/v3/users/cf1fd0d089e34556b6f6a90b6f5c74d4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:10.100069 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:10.101610 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:10.120706 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a094b34-245f-4b05-9af7-c45da94820a1 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:10.168168 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:10.178641 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:10.179498 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:10.179778 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:10.180051 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] PATH_INFO: `/v3/projects/6714c4681f0a4751bf63e59f55b45b05` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:10.181239 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6714c4681f0a4751bf63e59f55b45b05)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:10.183187 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:10.227543 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6714c4681f0a4751bf63e59f55b45b05', 'request_id': 'req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:10.233422 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6714c4681f0a4751bf63e59f55b45b05', 'request_id': 'req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:10.240650 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6714c4681f0a4751bf63e59f55b45b05', 'request_id': 'req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:10.251819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] The token cache is being invalidate because project 6714c4681f0a4751bf63e59f55b45b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 15:58:10.252110 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:10.252768 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 15:58:10.254281 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1441/2919] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 15:58:10 2022] DELETE /identity/v3/projects/6714c4681f0a4751bf63e59f55b45b05 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 15:58:19.835723 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:20.248913 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=27,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:58:39.631059 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:58:39.632519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:39.713684 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b97d7-f3de-4739-b487-f7c59015e962 admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:58:39.772554 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:58:39.773312 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:58:39.773595 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:58:39.773869 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:58:39.775639 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:58:39.787190 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:58:39.788902 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:58:39.798764 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1479/2920] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:58:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:58:49.794490 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=63 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:59:08.025742 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:59:08.027284 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:08.096133 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2f9f1-ccce-40a3-a7f1-a37d83c11901 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:59:08.100774 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:59:08.104226 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:59:08.104554 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:59:08.104968 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:59:08.107033 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:08.121877 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:59:08.123495 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389bde56-f8bd-463d-8344-2202fcc6389d service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:59:08.133254 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1442/2921] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:59:08.214559 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:59:08.223933 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c155216-ef97-48e5-9c50-95e559a539ed service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:59:08.228343 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:59:08.229192 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:59:08.229479 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:59:08.229777 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:59:08.231553 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:08.301656 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:59:08.303118 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:59:08.313990 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1480/2922] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 15:59:18.129515 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=26 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:59:18.309145 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=34 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:59:37.521838 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] REQUEST_METHOD: `POST` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:59:37.522283 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:59:37.522435 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:59:37.539667 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.auth.core [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] MFA Rules not processed for user `44b73e975868496fb800c68e2d007faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80365) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 15:59:37.540883 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:37.596748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1443/2923] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 15:59:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 15:59:41.068831 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 15:59:41.070795 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:41.085534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67969f8-7119-4e94-99be-2f47251a2a24 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 15:59:41.091973 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 15:59:41.092818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 15:59:41.093144 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 15:59:41.093421 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 15:59:41.095302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 15:59:41.109019 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 15:59:41.110546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 15:59:41.123561 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1481/2924] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 15:59:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 15:59:47.590709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=25 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 15:59:51.118066 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=6 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:00:39.884173 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:39.885709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:39.911350 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359a76b-37b1-4c10-ad99-b03ff06bffea None None] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:39.917559 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:39.918155 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:39.918481 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:39.918709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:39.920535 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:39.934917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:39.936177 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:39.949517 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1444/2925] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 16:00:40.073293 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:40.084411 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e05b348-c34e-404a-9b54-242108fd7860 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:40.087988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:40.088672 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:40.088956 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:40.089231 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:40.091345 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:40.106250 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:40.107694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:40.119307 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1482/2926] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:41.297931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:41.299436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:41.311109 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068f486-168e-4be8-8625-829751d4efe2 service nova] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:41.314748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:41.315368 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:41.315653 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:41.315931 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:41.317678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:41.331229 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:41.332720 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:41.344482 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1445/2927] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:42.163041 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:42.173349 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502428d1-73a0-4103-a179-96f109ff75b6 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:42.177264 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:42.177915 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:42.178197 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:42.178471 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:42.180225 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:42.240630 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:42.242063 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:42.252027 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1483/2928] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:46.012630 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:46.021240 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f359970-38c8-4e4a-bd35-2e94aea42f8c service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:46.024833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:46.025469 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:46.025748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:46.026022 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:46.027770 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:46.092864 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:46.094326 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:46.103906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1446/2929] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:49.127876 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:49.129362 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:49.144202 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce894870-023f-4491-a02e-242d29057efd service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:49.149098 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:49.149742 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:49.150020 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:49.150291 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:49.152007 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:49.165814 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:49.167482 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:49.178995 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1484/2930] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:51.957668 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:51.972802 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12497a-af53-4902-89b4-08cb7de921ad service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:51.980787 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:51.982451 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:51.983016 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:51.983579 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:51.986078 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:52.001849 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:52.003683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:52.026598 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1447/2931] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:00:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:00:52.032782 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:52.034765 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:52.058169 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30272285-dc3d-4ce0-88c3-fda5bbdf9095 service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:52.072006 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:52.072928 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:52.073214 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:52.073486 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] PATH_INFO: `/v3/users/63d864e3821b4c989b474bd313722f33` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:52.074710 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d864e3821b4c989b474bd313722f33)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:52.076762 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:52.125983 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d864e3821b4c989b474bd313722f33'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.126637 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d864e3821b4c989b474bd313722f33'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.127642 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d864e3821b4c989b474bd313722f33'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.135647 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d864e3821b4c989b474bd313722f33'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.143422 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d864e3821b4c989b474bd313722f33'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.161725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1485/2932] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 16:00:52 2022] DELETE /identity/v3/users/63d864e3821b4c989b474bd313722f33 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:00:52.337299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:52.354020 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22b8d4-331e-4fce-b61d-fff2dc0356b4 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:52.413140 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:52.414002 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:52.414282 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:52.414557 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] PATH_INFO: `/v3/projects/bc1fbb8218ad4da5a2e2ac6d50bf89e0` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:52.415724 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc1fbb8218ad4da5a2e2ac6d50bf89e0)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:52.417692 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:52.462356 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc1fbb8218ad4da5a2e2ac6d50bf89e0', 'request_id': 'req-719450d1-ef5c-405e-95b5-e47aa4672fbc'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.466996 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:00:52.468406 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc1fbb8218ad4da5a2e2ac6d50bf89e0', 'request_id': 'req-719450d1-ef5c-405e-95b5-e47aa4672fbc'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.476362 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc1fbb8218ad4da5a2e2ac6d50bf89e0', 'request_id': 'req-719450d1-ef5c-405e-95b5-e47aa4672fbc'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.487870 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] The token cache is being invalidate because project bc1fbb8218ad4da5a2e2ac6d50bf89e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 16:00:52.488158 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.488833 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.490382 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1448/2933] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 16:00:52 2022] DELETE /identity/v3/projects/bc1fbb8218ad4da5a2e2ac6d50bf89e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:00:52.494589 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:52.496104 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:52.509675 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f95c4d-17eb-4fab-b2c2-3eda2818674d admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:52.550226 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:00:52.566436 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:52.567302 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:52.567575 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:52.567842 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] PATH_INFO: `/v3/users/989d6807e2404d759f89a6b91c77e3f0` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:52.569061 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=989d6807e2404d759f89a6b91c77e3f0)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:52.570958 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:52.615994 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '989d6807e2404d759f89a6b91c77e3f0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.616588 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '989d6807e2404d759f89a6b91c77e3f0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.617195 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '989d6807e2404d759f89a6b91c77e3f0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.626219 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '989d6807e2404d759f89a6b91c77e3f0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.633976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-87629213-c5fb-49db-8d3b-f11cabd14c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '989d6807e2404d759f89a6b91c77e3f0'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.651081 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1486/2934] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 16:00:52 2022] DELETE /identity/v3/users/989d6807e2404d759f89a6b91c77e3f0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:00:52.819238 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:00:52.820844 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:00:52.835723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719450d1-ef5c-405e-95b5-e47aa4672fbc admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:00:52.897871 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:00:52.898678 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:00:52.899068 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:00:52.899343 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] PATH_INFO: `/v3/projects/b924dd4792284fad81f0acaef7083f9b` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:00:52.900558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b924dd4792284fad81f0acaef7083f9b)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:00:52.902502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:00:52.947225 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b924dd4792284fad81f0acaef7083f9b', 'request_id': 'req-48e298b7-b68a-4db7-9988-6890357a2b6e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.952599 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b924dd4792284fad81f0acaef7083f9b', 'request_id': 'req-48e298b7-b68a-4db7-9988-6890357a2b6e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.960455 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b924dd4792284fad81f0acaef7083f9b', 'request_id': 'req-48e298b7-b68a-4db7-9988-6890357a2b6e'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.971558 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] The token cache is being invalidate because project b924dd4792284fad81f0acaef7083f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80365) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 16:00:52.971839 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.972502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:00:52.974004 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1449/2935] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 16:00:52 2022] DELETE /identity/v3/projects/b924dd4792284fad81f0acaef7083f9b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:01:02.647635 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=33,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:01:02.968658 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:01:39.132185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] REQUEST_METHOD: `POST` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:01:39.132714 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:01:39.132810 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:01:39.150152 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.auth.core [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] MFA Rules not processed for user `44b73e975868496fb800c68e2d007faa`. Rule list: `[]` (Enabled: `True`). {{(pid=80366) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 16:01:39.151344 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:39.204026 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1487/2936] 10.222.0.45 () {58 vars in 957 bytes} [Tue May 24 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 16:01:39.907398 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:01:39.908813 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:39.966457 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e298b7-b68a-4db7-9988-6890357a2b6e admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:01:39.969991 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:01:39.970627 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:01:39.970906 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:01:39.971179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:01:39.972918 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:39.982967 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:01:39.984470 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:01:39.992394 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1450/2937] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:01:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:01:40.220979 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:01:40.223318 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:40.303517 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d6dc-c0bc-4b0f-8026-a2211b50ae87 None None] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:01:40.308474 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:01:40.309148 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:01:40.309429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:01:40.309731 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:01:40.311514 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:40.323351 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:01:40.325546 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:01:40.336146 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1488/2938] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:01:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:01:42.097407 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:01:42.106291 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffde7d-d4eb-42f6-8190-94358f798f08 service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:01:42.109917 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:01:42.110552 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:01:42.110984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:01:42.111262 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:01:42.113143 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:42.125230 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:01:42.126723 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:01:42.135861 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1451/2939] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:01:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:01:50.331832 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=57 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:01:52.131615 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=32 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:01:54.652913 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:01:54.654381 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:54.732564 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090f4f5-41bf-42c9-acb8-8643e4e709ee service placement] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:01:54.737950 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aa1fb96699054e5c94fcc188fd9535be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:01:54.738598 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:01:54.738878 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:01:54.739149 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:01:54.740926 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:01:54.753359 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:01:54.754912 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:01:54.765788 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1489/2940] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:01:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:02:04.760870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=34 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:03:41.285709 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:41.287093 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:41.332166 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7600ee2d-ca27-4af2-b92f-c84c90cd18bb service neutron] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:41.337102 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:41.337740 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:41.338024 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:41.338299 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:41.340748 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:41.352313 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:41.353914 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:41.363819 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1452/2941] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:03:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:03:44.788049 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:44.789518 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:44.801632 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118177c-7a8a-4240-b621-08de0f9f5e83 service nova] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:44.805333 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c28d40778004a2d8f2b8e2cb38a1f12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:44.805976 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] REQUEST_METHOD: `GET` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:44.806259 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:44.806534 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:44.808239 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:44.871222 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:44.872694 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:44.882818 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1490/2942] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:03:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:03:51.104101 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:51.105597 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:51.122210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7110-54f5-41e8-92d2-1ef357e04285 service placement] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:51.131436 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:51.132333 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:51.132644 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:51.132919 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] PATH_INFO: `/v3/users/44b73e975868496fb800c68e2d007faa` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:51.134106 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b73e975868496fb800c68e2d007faa)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:51.136035 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:51.187114 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b73e975868496fb800c68e2d007faa'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.187765 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b73e975868496fb800c68e2d007faa'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.188404 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b73e975868496fb800c68e2d007faa'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.195502 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b73e975868496fb800c68e2d007faa'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.207310 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b73e975868496fb800c68e2d007faa'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.220947 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1453/2943] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 16:03:51 2022] DELETE /identity/v3/users/44b73e975868496fb800c68e2d007faa => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:03:51.395489 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:51.405803 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4af7fe-83b3-4d4f-9da1-f14ead080864 service neutron] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:51.463847 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:51.464754 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:51.465045 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:51.465322 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] PATH_INFO: `/v3/projects/f5b185a216134b4d828add9ebdcf89d2` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:51.466493 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b185a216134b4d828add9ebdcf89d2)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:51.468429 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:51.516687 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b185a216134b4d828add9ebdcf89d2', 'request_id': 'req-92477bd9-80e1-459d-bb93-295271a99c0f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.522715 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b185a216134b4d828add9ebdcf89d2', 'request_id': 'req-92477bd9-80e1-459d-bb93-295271a99c0f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.529817 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b185a216134b4d828add9ebdcf89d2', 'request_id': 'req-92477bd9-80e1-459d-bb93-295271a99c0f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.542560 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] The token cache is being invalidate because project f5b185a216134b4d828add9ebdcf89d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 16:03:51.542870 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.543568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.545213 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1491/2944] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 16:03:51 2022] DELETE /identity/v3/projects/f5b185a216134b4d828add9ebdcf89d2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:03:51.549194 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:51.550565 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:51.563523 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a41b5a-2c8f-4db6-ba62-56bce9f90a09 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:51.618456 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:51.619252 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:51.619532 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:51.619806 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] PATH_INFO: `/v3/users/aff96157516f4c07b6d24d8081395ddf` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:51.620993 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff96157516f4c07b6d24d8081395ddf)` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:51.623164 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:51.655549 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:03:51.668975 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff96157516f4c07b6d24d8081395ddf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.669608 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff96157516f4c07b6d24d8081395ddf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.670210 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff96157516f4c07b6d24d8081395ddf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.677031 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff96157516f4c07b6d24d8081395ddf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.688820 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.notifications [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff96157516f4c07b6d24d8081395ddf'} {{(pid=80365) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:51.701179 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1454/2945] 10.222.0.45 () {58 vars in 1223 bytes} [Tue May 24 16:03:51 2022] DELETE /identity/v3/users/aff96157516f4c07b6d24d8081395ddf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:03:51.875356 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Authenticating user token {{(pid=80366) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:03:51.877391 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.fernet_utils [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80366) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:03:51.890185 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92477bd9-80e1-459d-bb93-295271a99c0f admin admin] Validating token access rules against request {{(pid=80366) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:03:51.911593 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:03:51.949425 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a944101ee294a37aa42cee1133a9032', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'afcdba301526446ba48fe1e8c0bb21bc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80366) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:03:51.950288 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:03:51.950568 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] SCRIPT_NAME: `/identity` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:03:51.950845 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] PATH_INFO: `/v3/projects/69cef26705864bda8ee1cd908c39542b` {{(pid=80366) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:03:51.952010 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69cef26705864bda8ee1cd908c39542b)` {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:03:51.953957 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] RBAC: Authorization granted {{(pid=80366) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:03:51.998519 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69cef26705864bda8ee1cd908c39542b', 'request_id': 'req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:52.004462 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69cef26705864bda8ee1cd908c39542b', 'request_id': 'req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:52.011476 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69cef26705864bda8ee1cd908c39542b', 'request_id': 'req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f'} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:52.022700 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] The token cache is being invalidate because project 69cef26705864bda8ee1cd908c39542b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80366) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 16:03:52.022988 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:52.023612 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG keystone.notifications [None req-b33da0f5-c42a-4673-8d6b-43fcf5d5330f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80366) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 16:03:52.025173 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: [pid: 80366|app: 0|req: 1492/2946] 10.222.0.45 () {58 vars in 1232 bytes} [Tue May 24 16:03:51 2022] DELETE /identity/v3/projects/69cef26705864bda8ee1cd908c39542b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 16:04:01.697524 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=17,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:04:02.019725 dv-cinder-843151-1-iscsi devstack@keystone.service[80366]: DEBUG dbcounter [-] [80366] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80366) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 16:08:47.896573 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Authenticating user token {{(pid=80365) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 16:08:47.897990 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:08:48.008277 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9af3cb-bb22-4ff7-993c-35a0fceab520 admin admin] Validating token access rules against request {{(pid=80365) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 16:08:48.066683 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ac56ca4fe95468784f955de52375920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6da052fcc124bd5b71972a9a0e094fa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80365) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 16:08:48.067363 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] REQUEST_METHOD: `GET` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 16:08:48.067648 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] SCRIPT_NAME: `/identity` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 16:08:48.067929 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80365) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 16:08:48.069727 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.fernet_utils [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80365) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 16:08:48.081389 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 16:08:48.082984 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27202d0b-2f88-42be-ad34-b8cf750b994d service placement] RBAC: Authorization granted {{(pid=80365) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 16:08:48.094439 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: [pid: 80365|app: 0|req: 1455/2947] 10.222.0.45 () {60 vars in 1365 bytes} [Tue May 24 16:08:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 16:08:58.089050 dv-cinder-843151-1-iscsi devstack@keystone.service[80365]: DEBUG dbcounter [-] [80365] Writing DB stats keystone:SELECT=76 {{(pid=80365) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}